The Pipeline Architecture
When building an AI evaluation system—such as an automated Applicant Tracking System (ATS) that scores resumes against a job rubric—performance and reliability are paramount. You cannot simply wrap a Claude API call in a synchronous Flask route. Large language models introduce high latency. If you process a resume synchronously on the main thread, your web server will block, timeout, and eventually crash under load.
A production-grade evaluation pipeline requires an asynchronous architecture. The standard stack for this is FastAPI (for high-concurrency API routing) paired with Celery (for distributed background task execution) and Redis (as the message broker).
The Endpoint Design
The FastAPI endpoint acts purely as a dispatcher.
When a frontend client uploads a resume PDF, the endpoint accepts the file, writes it to secure storage (e.g., an S3 bucket or Supabase Storage), and immediately hands the Job ID and File URL off to the Celery broker.
The endpoint responds in milliseconds with a 202 Accepted status and a task_id. The frontend uses this task_id to poll a secondary /status endpoint, updating the UI with a progress spinner while the heavy lifting occurs in the background.
OCR Preprocessing
Inside the Celery worker, the first operation is not calling the LLM. It is OCR extraction.
Passing a raw PDF directly to an LLM's vision capabilities is incredibly inefficient. It consumes massive amounts of tokens and often hallucinates when encountering complex multi-column resume layouts.
Instead, the Python worker downloads the PDF and runs it through a robust OCR library (like pdfplumber or pytesseract). This enforces standardization. By extracting the raw UTF-8 text, we sanitize the input, strip out useless graphical elements, and drastically reduce the token payload sent to the Claude API.
Prompt Construction and JSON Parsing
The core of the evaluation logic relies on strict prompt engineering. To integrate an LLM into a software pipeline, it must return structured data, not conversational text.
We utilize Claude 3.5 Haiku. Haiku is chosen specifically because evaluation tasks require high reading comprehension but low creative generation. Haiku is significantly cheaper and faster than Sonnet or Opus while maintaining the reasoning capability required to grade a resume.
The prompt must explicitly demand JSON:
prompt = f"""
Evaluate the following candidate text against the rubric.
RUBRIC: {json.dumps(rubric)}
CANDIDATE TEXT: {extracted_text}
You MUST return your evaluation as a valid JSON object. Do not include markdown formatting or introductory text.
{{
"score": <int>,
"reasoning": "<string>"
}}
"""
When the Claude API returns the response, the Python worker parses it using json.loads().
Handling Error States
LLMs are non-deterministic. Even with strict prompting, Claude might occasionally wrap the JSON in markdown code blocks (```json), which will crash the Python json.loads() parser.
The pipeline must include a resilient parsing layer that utilizes regex to strip markdown formatting before attempting to parse the string. If the parsing still fails, the Celery task must catch the JSONDecodeError, log the failure to a monitoring tool (like Sentry), and transition the database record to an error state rather than allowing the worker to crash silently.