Posted 18 days ago
AI Conversation Research & Evaluation Specialist
@ Gramian Consulting Group
Remote in Canada
Overview
Gramian Consulting Group is hiring an AI Conversation Research & Evaluation Specialist, a subject matter expert role to review conversations between people and AI models. The work involves judging whether enough context was gathered before recommendations were made and suggesting follow-up questions. Candidates need 3–5 years of relevant experience in social work, counseling or psychology, qualitative research or HCI, or advisory fields.
Responsibilities:
- Review AI conversations about interpersonal situations and everyday decisions
- Identify missing context and recommend follow-up questions
- Distinguish decision-critical information from non-essential details
- Conduct qualitative reviews using structured guidelines
- Collaborate with research teams for consistent assessments
Requirements summary:
- 3–5 years of direct professional experience in a qualifying domain
- Licensed social work, counseling, or psychology credential, or MA/PhD in qualitative research or HCI
- Experience in client assessment, interviewing, or qualitative analysis
- Strong analytical and written communication skills
Tools & skills
Source and dates
Employer career site / ATS feed · workable
- Employer posting date
- 2026-10-21
- First seen by our source
- 2026-10-21
- Source record updated
- 2026-10-21
These dates describe the source record, not a live verification that the employer is still hiring. Check the original posting.
Job details
| Seniority | Mid level |
| Department | Research & science |
| Experience | 3+ years |
| Travel | air: none · ground: none |
| Work setting | Office |
| Position | Mostly sitting |
| Physical demands | physical low · computer high |
| Languages | English |
| Licences & certifications | LCSW, LICSW, LMSW, LMFT, LPC, LCPC |
| Education | Master’s degree (preferred) |
| Fields of study | Social Work, Psychology, Counseling, Human-Computer Interaction, Communication |
Summary and details written by AI from the posting; the employer’s posting is the authority.
Gramian Consulting Group
Saved company profile
Historical company information from a saved The Jobs API profile (LinkedIn/Crunchbase fields). It is not live-verified; figures may have changed. Missing values remain unknown.
We get talents. You get results.
Full company description
Gramian Consulting is your partner for accessing the engineering capabilities you need—delivered in the model that fits your business, from staff augmentation and talent recruiting to Build-Operate-Transfer (BOT). We combine the perspective of a software engineer, the rigor of a technical recruiter, and the vision of a business builder, so you get experts who understand your challenges and deliver results the right way. This blend is our signature advantage in providing top-quality services, fast and reliably.
- LinkedIn followers
- 27,283
- People associated on LinkedIn
- 7
- Reported office locations
- Kumanovo, MK
- Other recorded names
- Gramian Consulting Group
- Gramian Consulting
- Specialties
- Talent Solutions
- Technical Recruiting
- Staff Augmentation
- AI & Data Solutions
- and BOT
- Profile identifies a recruitment agency
- No
Gramian Consulting is your partner for accessing the engineering capabilities you need—delivered in the model that fits your business, from staff augmentation and talent recruiting to Build-Operate-Transfer (BOT). We combine the perspective of a software engineer, the rigor of a technical recruiter, and the vision of a business builder, so you get experts who understand your challenges and deliver results the right…
| Field | Value |
|---|---|
| Industry | IT Services and IT Consulting |
| Company size | 2-10 employees |
| Headquarters | Kumanovo |
| Organization type | Partnership |
| Year founded | 2025 |
| Specialties | Talent SolutionsTechnical RecruitingStaff AugmentationAI & Data Solutionsand BOT |
| Open jobs on Jobsy | 46 |
| View page |
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are seeking experienced Subject Matter Experts (SMEs)in psychology, social work, counseling, qualitative research, human-computer interaction (HCI), communication, conflict resolution, coaching, and related advisory fields to support AI conversation evaluation research. You will review conversations between people and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. The ideal candidate has at least 3–5 years of relevant professional experience, strong analytical skills, and the ability to distinguish information that could materially change a recommendation from details that are less relevant.
Key Responsibilities
Review AI conversations involving real-world interpersonal situations and everyday decision-making.
Identify missing context and recommend relevant follow-up questions that could improve the quality of AI responses.
Evaluate whether gathering additional information would materially change an AI-generated recommendation.
Distinguish between load-bearing information that affects a decision and non-essential or merely descriptive details.
Conduct qualitative reviews and document findings using structured evaluation guidelines.
Assess whether conversations adequately consider relevant personal, psychological, social, financial, practical, and organizational factors.
Identify missed opportunities for clarification, deeper understanding, and information gathering.
Review conversations involving relationships, work, education, family, wellbeing, planning, and resource constraints.
Collaborate with research teams to ensure consistent, evidence-based assessments.
Maintain high standards of accuracy, clarity, and consistency while meeting project deadlines.
Candidates must have at least 3–5 years of relevant direct professional experience in one or more of the following domains:
1. Licensed Social Work
- Master's degree in Social Work (MSW) or an equivalent graduate-level qualification.
- LCSW, LICSW, LMSW, or an equivalent applicable social work license or credential.
- Experience in client assessment, case review, interviewing, counseling, or related social-work practice.
2. Counseling, Therapy, or Psychology
- Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling or therapy license or credential; or
- PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology.
- Relevant experience in counseling, therapy, psychological assessment, interviewing, case review, or other client-facing professional work.
3. Qualitative Research, HCI, or Communication
- MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline.
- Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding.
- Ability to analyze conversations systematically and identify missing context, communication patterns, and opportunities for clarification.
4. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields
- Relevant professional qualification, credential, or demonstrated subject-matter expertise.
- At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway.
- Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work.
- Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations.
Journal
Only you can see this. Kept on this device, and in your account when you sign in.