Custom GPT Assessment Workflow

This recipe shows you how to create a job, search for candidates, then use OpenAI’s API directly (like our internal system) to generate custom fit scores and outreach emails with your own prompts and logic.

Overview

What you’ll accomplish:
  1. Create a job description for candidate matching
  2. Search for developers using AI-powered search
  3. Build custom GPT-4 assessment using OpenAI API directly
  4. Generate personalized fit scores and email templates
  5. Customize the assessment logic for your specific needs
Total time: ~10 minutes
Credits used: 1 per candidate searched (no assessment credits - you’re using your own OpenAI API)

Step 1: Setup and Job Creation

First, set up your environment with both B2D and OpenAI API keys.
import requests
import json
from openai import OpenAI
from typing import Dict, List, Optional
import os

# Setup API clients
B2D_API_KEY = "YOUR_B2D_API_KEY"
OPENAI_API_KEY = "YOUR_OPENAI_API_KEY"

openai_client = OpenAI(api_key=OPENAI_API_KEY)

b2d_headers = {
    "x-api-key": B2D_API_KEY,
    "Content-Type": "application/json"
}

# Create job description
job_data = {
    "description": """Lead Frontend Engineer - React/TypeScript

We're looking for a senior frontend engineer to lead our product team at a Series B fintech startup.

Requirements:
- 7+ years frontend development experience
- Expert-level React and TypeScript skills
- Experience with state management (Redux, Zustand, etc.)
- Design system and component library experience
- Team leadership and mentoring experience
- Fintech or financial services background preferred

Responsibilities:
- Lead frontend architecture decisions
- Mentor junior developers
- Build complex financial UIs and data visualizations
- Collaborate with product and design teams
- Establish frontend best practices and standards

What we offer:
- $180k-220k + significant equity
- Remote-first with NYC office access
- Technical leadership opportunities
- Cutting-edge fintech product"""
}

print("📝 Creating job description...")
job_response = requests.post("https://api.b2d.ai/jobs/create", 
                            headers=b2d_headers, json=job_data)

if job_response.status_code == 200:
    job_id = job_response.json()['id']
    print(f"✅ Job created: {job_id}")
else:
    print(f"❌ Error: {job_response.json()}")
    exit()

Step 2: Search for Candidates

Search for developers who match your job requirements.
# Search parameters
search_params = {
    "size": 15,  # Get 15 candidates for assessment
    "countries": ["United States", "Canada"],
    "required_fields": ["email", "full_name"],
    "summary_length": 3000  # Longer summaries for better assessment
}

print("\n🔍 Searching for candidates...")
search_response = requests.get(
    f"https://api.b2d.ai/jobs/{job_id}/search",
    headers={"x-api-key": B2D_API_KEY},
    params=search_params
)

if search_response.status_code == 200:
    candidates = search_response.json()['results']
    print(f"✅ Found {len(candidates)} candidates")
    
    # Show preview of candidates
    for i, candidate in enumerate(candidates[:3], 1):
        name = candidate.get('full_name', 'N/A')
        username = candidate['github_username']
        company = candidate.get('company', 'N/A')
        followers = candidate.get('followers', 0)
        
        print(f"  {i}. {name} (@{username}) - {company} | {followers} followers")
        
else:
    print(f"❌ Search failed: {search_response.json()}")
    exit()

print(f"💳 Search credits used: {len(candidates)}")

Step 3: Custom GPT Assessment Function

Create your own assessment function using OpenAI’s API, based on our internal implementation.
def assess_candidate_with_custom_gpt(candidate: Dict, job_description: str, 
                                   additional_context: str = "") -> Dict:
    """
    Custom GPT-4 assessment function similar to B2D's internal implementation.
    Returns fit score, hireability, fit_summary, and email_copy.
    """
    
    # Build candidate summary for GPT
    candidate_summary = {
        "github_username": candidate.get('github_username'),
        "full_name": candidate.get('full_name'),
        "bio": candidate.get('bio'),
        "company": candidate.get('company'),
        "location": candidate.get('location'),
        "followers": candidate.get('followers'),
        "public_repos": candidate.get('public_repos'),
        "summary_text": candidate.get('summary_text', ''),  # GitHub activity summary
        "top_repositories": candidate.get('repos', [])[:5],  # Top 5 repos
        "recent_commits": candidate.get('commits', [])[:3],  # Recent commits
        "skills": candidate.get('skills', [])[:10],  # Top skills
    }
    
    # Custom system prompt - you can modify this for your specific needs
    system_prompt = f"""You are an expert technical recruiter evaluating candidates for engineering roles.

Given a job description and a candidate's GitHub profile data, you must assess their fit and generate outreach.

Job Description:
{job_description}

Additional Context: {additional_context}

You must return a JSON object with exactly these fields:
- "fit": integer 0-10 (how well their GitHub activity matches the job requirements)
- "fit_summary": string (1-2 sentences explaining fit, referencing specific repos/technologies)
- "hireability": integer 0-10 (overall GitHub presence and professional quality)
- "email_copy": string (2-3 sentence personalized outreach message referencing their specific work)

Focus on:
- Relevant technologies and frameworks from their repositories
- Project complexity and impact (stars, forks, contributors)
- Contribution patterns and code quality
- Leadership indicators (maintained projects, documentation)
- Recent activity and engagement

For email_copy, be specific about their projects and how they relate to the role. Avoid generic praise.

Return only valid JSON."""

    try:
        # Call GPT-4 (using same model as B2D's internal system)
        response = openai_client.chat.completions.create(
            model="gpt-4o-mini",  # Same model B2D uses internally
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": json.dumps(candidate_summary, indent=2)}
            ],
            temperature=0.7,
            max_tokens=800
        )
        
        # Parse JSON response
        assessment_text = response.choices[0].message.content.strip()
        
        # Handle potential JSON parsing issues
        if assessment_text.startswith("```json"):
            assessment_text = assessment_text.replace("```json", "").replace("```", "").strip()
        
        assessment = json.loads(assessment_text)
        
        # Validate required fields
        required_fields = ['fit', 'fit_summary', 'hireability', 'email_copy']
        for field in required_fields:
            if field not in assessment:
                raise ValueError(f"Missing required field: {field}")
        
        # Log token usage for cost tracking
        usage = response.usage
        print(f"  💰 OpenAI tokens used: {usage.total_tokens} (${usage.total_tokens * 0.00015:.4f})")
        
        return assessment
        
    except json.JSONDecodeError as e:
        print(f"  ❌ JSON parsing error: {e}")
        return {"fit": 0, "fit_summary": "Assessment failed", "hireability": 0, "email_copy": "Error"}
    except Exception as e:
        print(f"  ❌ Assessment error: {e}")
        return {"fit": 0, "fit_summary": "Assessment failed", "hireability": 0, "email_copy": "Error"}

Step 4: Run Custom Assessments

Assess your candidates using your custom GPT function.
print("\n🤖 Running custom GPT assessments...")

assessments = []
total_tokens = 0
job_desc = job_data["description"]

# Additional context for better assessments
context = "Series B fintech startup, remote-first, equity package, technical leadership role"

for i, candidate in enumerate(candidates, 1):
    username = candidate['github_username']
    name = candidate.get('full_name', username)
    
    print(f"  🔍 Assessing {i}/{len(candidates)}: {name} (@{username})")
    
    # Run custom assessment
    assessment = assess_candidate_with_custom_gpt(
        candidate=candidate,
        job_description=job_desc,
        additional_context=context
    )
    
    # Combine candidate data with assessment
    full_result = {
        **candidate,  # Include all original candidate data
        **assessment  # Add assessment results
    }
    
    assessments.append(full_result)
    
    # Show results
    if assessment['fit'] > 0:  # Valid assessment
        print(f"    🎯 Fit: {assessment['fit']}/10 | ⭐ Hire: {assessment['hireability']}/10")
        print(f"    📝 {assessment['fit_summary'][:100]}...")
    
print(f"\n✅ Completed {len(assessments)} custom assessments")
Expected output:
🤖 Running custom GPT assessments...
  🔍 Assessing 1/15: Emily Rodriguez (@emily-react-dev)
    💰 OpenAI tokens used: 1247 ($0.0019)
    🎯 Fit: 9/10 | ⭐ Hire: 8/10
    📝 Excellent match with React expertise shown in react-fintech-ui and typescript-design-system...

  🔍 Assessing 2/15: Michael Chen (@mchen-frontend)
    💰 OpenAI tokens used: 1156 ($0.0017)
    🎯 Fit: 7/10 | ⭐ Hire: 9/10
    📝 Strong frontend skills with Redux experience in trading-dashboard repo, though limited fintech...

✅ Completed 15 custom assessments

Step 5: Analyze and Rank Results

Sort candidates by fit score and generate your hiring pipeline.
# Sort by fit score (highest first)
top_candidates = sorted(assessments, key=lambda x: x['fit'], reverse=True)

print("\n🏆 Top 10 Candidates by Fit Score:")
print("=" * 80)

for i, candidate in enumerate(top_candidates[:10], 1):
    name = candidate.get('full_name', candidate['github_username'])
    username = candidate['github_username']
    email = candidate.get('email', 'N/A')
    fit = candidate['fit']
    hire = candidate['hireability']
    location = candidate.get('location', 'N/A')
    
    print(f"{i:2d}. {name} (@{username})")
    print(f"    📧 {email}")
    print(f"    📍 {location}")
    print(f"    🎯 Job Fit: {fit}/10 | ⭐ Hireability: {hire}/10")
    print(f"    📝 Assessment: {candidate['fit_summary']}")
    print(f"\n    📨 Outreach Email:")
    print(f"    \"{candidate['email_copy']}\"\n")
    print("-" * 80)

# Show filtering options
high_fit_candidates = [c for c in assessments if c['fit'] >= 8]
contactable_candidates = [c for c in assessments if c.get('email') and c['fit'] >= 7]

print(f"\n📊 Pipeline Summary:")
print(f"   • Total candidates assessed: {len(assessments)}")
print(f"   • High fit (8+ score): {len(high_fit_candidates)} candidates")
print(f"   • High fit + contactable: {len(contactable_candidates)} candidates")

Step 6: Advanced Custom Scoring

Add your own custom scoring logic on top of GPT assessment.
def calculate_custom_score(candidate: Dict) -> Dict:
    """Add custom scoring logic based on your specific criteria"""
    
    base_fit = candidate['fit']
    base_hire = candidate['hireability']
    
    # Custom scoring factors
    bonus_score = 0
    factors = []
    
    # Bonus for high GitHub activity
    followers = candidate.get('followers', 0)
    if followers > 1000:
        bonus_score += 1
        factors.append(f"High influence ({followers} followers)")
    
    # Bonus for recent activity
    recent_commits = candidate.get('commits', [])
    if len(recent_commits) >= 3:
        bonus_score += 1
        factors.append("Recent activity")
    
    # Bonus for popular repositories
    repos = candidate.get('repos', [])
    popular_repos = [r for r in repos if r.get('stargazers_count', 0) > 100]
    if popular_repos:
        bonus_score += 1
        factors.append(f"{len(popular_repos)} popular repos")
    
    # Bonus for fintech/finance keywords
    bio = candidate.get('bio', '').lower()
    summary = candidate.get('summary_text', '').lower()
    fintech_keywords = ['fintech', 'finance', 'trading', 'payments', 'banking']
    
    if any(keyword in bio + summary for keyword in fintech_keywords):
        bonus_score += 2
        factors.append("Fintech experience")
    
    # Calculate adjusted scores (max 10)
    adjusted_fit = min(base_fit + bonus_score, 10)
    priority_score = (adjusted_fit * 0.7) + (base_hire * 0.3)  # Weight fit more heavily
    
    return {
        'original_fit': base_fit,
        'adjusted_fit': adjusted_fit,
        'hireability': base_hire,
        'priority_score': round(priority_score, 1),
        'bonus_factors': factors,
        'bonus_points': bonus_score
    }

# Apply custom scoring
print("\n🎯 Applying Custom Scoring Logic...")

for candidate in assessments:
    custom_scores = calculate_custom_score(candidate)
    candidate.update(custom_scores)

# Re-sort by priority score
priority_ranked = sorted(assessments, key=lambda x: x['priority_score'], reverse=True)

print("\n🏅 Top 5 by Custom Priority Score:")
for i, candidate in enumerate(priority_ranked[:5], 1):
    name = candidate.get('full_name', candidate['github_username'])
    username = candidate['github_username']
    
    print(f"{i}. {name} (@{username})")
    print(f"   🎯 Priority Score: {candidate['priority_score']}/10")
    print(f"   📈 Fit: {candidate['original_fit']}{candidate['adjusted_fit']} (+{candidate['bonus_points']})")
    print(f"   ⭐ Hireability: {candidate['hireability']}/10")
    
    if candidate['bonus_factors']:
        print(f"   🎁 Bonus factors: {', '.join(candidate['bonus_factors'])}")
    print()

Step 7: Export and Integration

Save results and integrate with your existing hiring workflow.
from datetime import datetime
import csv

# Generate comprehensive report
report = {
    "workflow_info": {
        "job_id": job_id,
        "date": datetime.now().isoformat(),
        "job_description": job_desc,
        "search_size": len(candidates),
        "assessments_completed": len([a for a in assessments if a['fit'] > 0]),
        "openai_model": "gpt-4o-mini",
        "total_b2d_credits": len(candidates)
    },
    "top_candidates": [
        {
            "rank": i + 1,
            "name": c.get('full_name'),
            "github_username": c['github_username'],
            "email": c.get('email'),
            "location": c.get('location'),
            "company": c.get('company'),
            "followers": c.get('followers'),
            "original_fit_score": c['original_fit'],
            "adjusted_fit_score": c['adjusted_fit'],
            "hireability_score": c['hireability'],
            "priority_score": c['priority_score'],
            "bonus_factors": c['bonus_factors'],
            "fit_summary": c['fit_summary'],
            "email_copy": c['email_copy'],
            "github_profile": f"https://github.com/{c['github_username']}"
        }
        for i, c in enumerate(priority_ranked[:10])
    ]
}

# Save JSON report
filename = f"custom_assessment_{job_id[:8]}_{datetime.now().strftime('%Y%m%d')}.json"
with open(filename, 'w') as f:
    json.dump(report, f, indent=2)

# Save CSV for spreadsheet import
csv_filename = f"candidates_{job_id[:8]}.csv"
with open(csv_filename, 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(['Rank', 'Name', 'GitHub', 'Email', 'Location', 'Company', 
                    'Priority Score', 'Fit Score', 'Hireability', 'Fit Summary'])
    
    for i, candidate in enumerate(priority_ranked[:10], 1):
        writer.writerow([
            i,
            candidate.get('full_name', ''),
            candidate['github_username'],
            candidate.get('email', ''),
            candidate.get('location', ''),
            candidate.get('company', ''),
            candidate['priority_score'],
            candidate['adjusted_fit'],
            candidate['hireability'],
            candidate['fit_summary'][:200] + '...'  # Truncate for CSV
        ])

print(f"📄 Detailed report saved: {filename}")
print(f"📊 CSV export saved: {csv_filename}")

# Cost summary
print(f"\n💰 Cost Summary:")
print(f"   B2D API credits used: {len(candidates)}")
print(f"   Estimated OpenAI cost: ${len(assessments) * 0.002:.2f}")  # Rough estimate
print(f"   Total cost: ~${0.01 * len(candidates) + len(assessments) * 0.002:.2f}")

Complete Custom Assessment Script

Here’s the complete script combining all steps:
#!/usr/bin/env python3
"""
Custom GPT Assessment Workflow
Combines B2D API search with direct OpenAI API assessment
"""

import requests
import json
from openai import OpenAI
from datetime import datetime
import csv
from typing import Dict, List

class CustomAssessmentWorkflow:
    def __init__(self, b2d_api_key: str, openai_api_key: str):
        self.b2d_api_key = b2d_api_key
        self.openai_client = OpenAI(api_key=openai_api_key)
        self.b2d_headers = {
            "x-api-key": b2d_api_key,
            "Content-Type": "application/json"
        }
    
    def create_job(self, job_description: str) -> str:
        """Create job and return job_id"""
        response = requests.post("https://api.b2d.ai/jobs/create", 
                                headers=self.b2d_headers, 
                                json={"description": job_description})
        
        if response.status_code != 200:
            raise Exception(f"Job creation failed: {response.json()}")
        
        return response.json()['id']
    
    def search_candidates(self, job_id: str, size: int = 20, 
                         countries: List[str] = None) -> List[Dict]:
        """Search for matching candidates"""
        params = {
            "size": size,
            "required_fields": ["email", "full_name"],
            "summary_length": 3000
        }
        
        if countries:
            params["countries"] = countries
        
        response = requests.get(f"https://api.b2d.ai/jobs/{job_id}/search",
                               headers={"x-api-key": self.b2d_api_key}, 
                               params=params)
        
        if response.status_code != 200:
            raise Exception(f"Search failed: {response.json()}")
        
        return response.json()['results']
    
    def assess_with_gpt(self, candidate: Dict, job_description: str, 
                       context: str = "") -> Dict:
        """Custom GPT assessment"""
        
        candidate_data = {
            "github_username": candidate.get('github_username'),
            "full_name": candidate.get('full_name'),
            "bio": candidate.get('bio'),
            "company": candidate.get('company'),
            "summary_text": candidate.get('summary_text', ''),
            "skills": candidate.get('skills', [])[:10],
            "repos": candidate.get('repos', [])[:5]
        }
        
        system_prompt = f"""You are an expert technical recruiter.

Job Description: {job_description}
Additional Context: {context}

Assess the candidate and return JSON with:
- "fit": integer 0-10 (job match score)
- "fit_summary": string (1-2 sentences with specific repos/technologies)  
- "hireability": integer 0-10 (overall GitHub quality)
- "email_copy": string (2-3 sentence personalized outreach)

Reference specific repositories and technologies. Be precise and actionable."""

        try:
            response = self.openai_client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {"role": "system", "content": system_prompt},
                    {"role": "user", "content": json.dumps(candidate_data, indent=2)}
                ],
                temperature=0.7,
                max_tokens=600
            )
            
            result = response.choices[0].message.content.strip()
            if result.startswith("```json"):
                result = result.replace("```json", "").replace("```", "").strip()
            
            return json.loads(result)
            
        except Exception as e:
            print(f"Assessment error for {candidate.get('github_username')}: {e}")
            return {"fit": 0, "fit_summary": "Error", "hireability": 0, "email_copy": "Error"}
    
    def run_complete_workflow(self, job_description: str, search_size: int = 15):
        """Run complete workflow and return results"""
        
        print("🚀 Starting Custom GPT Assessment Workflow\n")
        
        # Step 1: Create job
        print("📝 Creating job...")
        job_id = self.create_job(job_description)
        print(f"✅ Job created: {job_id}")
        
        # Step 2: Search candidates
        print(f"\n🔍 Searching for {search_size} candidates...")
        candidates = self.search_candidates(job_id, size=search_size, 
                                          countries=["United States", "Canada"])
        print(f"✅ Found {len(candidates)} candidates")
        
        # Step 3: Assess with custom GPT
        print(f"\n🤖 Running GPT assessments...")
        assessments = []
        
        for i, candidate in enumerate(candidates, 1):
            username = candidate['github_username']
            name = candidate.get('full_name', username)
            print(f"  {i}/{len(candidates)}: {name}")
            
            assessment = self.assess_with_gpt(candidate, job_description)
            
            if assessment['fit'] > 0:
                combined = {**candidate, **assessment}
                assessments.append(combined)
                print(f"    ✅ Fit: {assessment['fit']}/10, Hire: {assessment['hireability']}/10")
            else:
                print(f"    ❌ Assessment failed")
        
        # Step 4: Sort and return results
        results = sorted(assessments, key=lambda x: x['fit'], reverse=True)
        
        print(f"\n🏆 Assessment Complete!")
        print(f"   Successfully assessed: {len(results)}")
        print(f"   Top candidate fit score: {results[0]['fit'] if results else 0}/10")
        
        return {
            "job_id": job_id,
            "job_description": job_description,
            "candidates_found": len(candidates),
            "successful_assessments": len(results),
            "top_candidates": results[:10],
            "all_results": results
        }

# Usage example
if __name__ == "__main__":
    # Initialize workflow
    workflow = CustomAssessmentWorkflow(
        b2d_api_key="YOUR_B2D_API_KEY",
        openai_api_key="YOUR_OPENAI_API_KEY"
    )
    
    # Define job
    job_desc = """Senior React Developer
    
Looking for an expert React developer with TypeScript experience...
[Your full job description]"""
    
    # Run workflow
    results = workflow.run_complete_workflow(job_desc, search_size=20)
    
    # Save results
    with open("custom_assessment_results.json", "w") as f:
        json.dump(results, f, indent=2)
    
    print("📄 Results saved to custom_assessment_results.json")

Key Advantages of Custom Assessment

Full Control: Customize prompts, scoring logic, and assessment criteria
Cost Effective: Use your OpenAI API directly (typically $0.002 per assessment)
Flexible Scoring: Add custom bonus factors and weighting
Enhanced Context: Include company-specific context and requirements
Integration Ready: Easy to integrate with existing hiring tools
Transparent: Full visibility into assessment logic and costs

Customization Options

  • Custom Prompts: Modify the system prompt for your industry/role
  • Scoring Logic: Add custom bonus factors and weighting systems
  • Assessment Criteria: Focus on specific technologies or experience types
  • Output Format: Generate additional fields like salary recommendations
  • Integration: Connect to your ATS, Slack, or email systems

Cost Comparison

MethodCost per AssessmentControl LevelCustom Logic
B2D Assessment API1 credit (~$0.01)LimitedNo
Custom GPT (this recipe)~$0.002FullYes