Jev Prompts & Question Design: Writing Instructions, Criteria, and Answer Spaces

Developers frequently search for "Jev prompts" expecting to write the kind of open-ended conversational prompts used with ChatGPT or Claude. But TypeSafe Jev does not have a prompt box. It strictly separates what to evaluate (passed in state) from how to evaluate it (defined in typed question schemas using instructions and criteria).

In Jev, "prompt engineering" is really answer-space engineering: defining mutually exclusive categories, explicit scoring rubrics, and unambiguous predicate statements. When questions are structured well, Jev delivers high confidence scores and clean probability distributions. When questions are ambiguous or overlapping, probability mass scatters across options.

The Golden Rule: Separate State from Questions

With traditional LLMs, developers often mix context and instructions into a single block of text: "Here is an email from Bob. Bob says he wants a refund. Please read the email and tell me if he is angry."

With Jev, this must be decoupled:

• state contains only the data to be evaluated (the email text, customer attributes, transaction logs).
• questions contains only the operational criteria and options.

If you put instructions inside state (e.g., "Please classify this ticket"), Jev treats those sentences as raw data to be judged, not instructions to execute.

---

1. Designing Choice Questions: Boundaries and Exclusivity

Choice selects one option from 2 to 255 discrete possibilities. The most common failure mode is defining options that semantically overlap, causing Jev to split probability evenly between them.

Weak Example: Vague, Overlapping Options

# POOR: Options are ambiguous and overlap
"department": Choice(
    instructions="Where should this support ticket go?",
    criteria={
        "customer_issue": "Problems reported by customers",
        "tech_support": "Technical bugs and errors",
        "billing": "Money and invoices",
        "urgent_tickets": "Tickets that need fast attention"
    }
)

Why this fails:

• An angry customer with a payment bug qualifies for "customer_issue", "tech_support", "billing", and "urgent_tickets" simultaneously.
• Jev distributes probability across all four (e.g. [0.28, 0.24, 0.26, 0.22]), returning low confidence on whichever happens to take first place.
• "urgent_tickets" is an orthogonal dimension (urgency) mixed into a categorical routing decision.

Improved Example: Mutually Exclusive Categories

# IMPROVED: Mutually exclusive categories with explicit boundary definitions
"department": Choice(
    instructions="Route ticket to the primary operational queue based on 'issue_description'.",
    criteria={
        "billing": "Invoices, disputed credit card charges, refund requests, or payment portal errors",
        "infrastructure": "Server outages, API 500 errors, webhook delivery failures, or downtime",
        "account_access": "Password resets, multi-factor authentication issues, or account lockouts",
        "other": "General inquiries or requests that do not match the above categories"
    }
)

Why this works:

• Each option describes concrete situations rather than broad adjectives.
• The inclusion of an "other" escape hatch prevents Jev from forcing out-of-scope tickets into an incorrect category.
• Urgency is extracted into its own separate question evaluated in parallel.

---

2. Designing Score Rubrics: Levels and Progression

Score evaluates state against an ordered scale of 2 to 10 tiers. The returned score value is a continuous float representing the expected position along the level indices ($0 \dots N-1$).

TypeSafe's jaggedness documentation explicitly notes that while levels are ordered, numerical calibration across levels is weak in jev-1.13: you should not assume uncertainty distributes exclusively to adjacent levels or rely on fractional scores as precise continuous measurements between rubric tiers. Clear, discrete criteria definitions remain essential.

Weak Example: Subjective Adjectives

# POOR: Subjective labels without behavioral definitions
"lead_score": Score(
    instructions="How good is this inbound lead?",
    levels=["bad", "okay", "good", "great"]
)

Why this fails:

• Jev has no objective anchor for what makes a lead "okay" versus "good".
• Decisions become noisy and sensitive to minor wording changes in the state.

Improved Example: Concrete Behavioral Tiers

# IMPROVED: Concrete criteria anchored to state attributes
"lead_score": Score(
    instructions="Grade inbound lead qualification based on 'company_size', 'budget', and 'role'.",
    levels=[
        "disqualified",  # Personal email, student, or no budget
        "nurture",       # Commercial company, but under 20 employees or non-decision maker
        "qualified",     # 20-250 employees, verified corporate domain, VP or Director title
        "high_priority"  # 250+ employees, budget > $50k, C-suite or VP decision maker
    ]
)

Why this works:

• Each level maps to objective attributes present in the state.
• If a lead has 200 employees and a VP title, Jev confidently selects "qualified" with high probability.

---

3. Designing Noul Questions: Avoiding Negative Inversions

Noul evaluates whether a single condition is true, returning a probability between 0.0 and 1.0.

Weak Example: Negated or Inverted Phrasing

# POOR: Double negative and inverted boolean logic
"is_not_invalid": Noul(
    instructions="Is this document not failing any policy checks?"
)

Why this fails:

• Jev is documented as a literal reader. Indirection, rhetorical framing, or double negatives increase cognitive ambiguity and risk misclassification.
• Interpreting what 0.85 means in application code becomes mentally confusing: does 0.85 mean it is valid or invalid?

Improved Example: Direct Affirmative Statements

# IMPROVED: Direct, positive predicate statement
"is_policy_compliant": Noul(
    instructions="Does the expense report in 'receipt_data' fully comply with travel and meal expenditure policies?"
)

Why this works:

• Clear, affirmative condition.
• noul: 0.95 cleanly translates to: "95% probability this expense is compliant."

---

Complete Multi-Question Pattern

Here is how you combine well-designed questions in a single production request:

from typesafe_sdk import TypeSafeClient, Choice, Score, Noul

client = TypeSafeClient()

state = {
    "ticket_id": "TCK-8812",
    "customer_tier": "enterprise",
    "message": "We deployed your new SDK version and all database queries are timing out. Our customer checkout is failing.",
    "mrr_value": 8500
}

response = client.system_one(
    model="jev-latest",
    state=state,
    questions={
        # Categorical routing
        "routing_target": Choice(
            instructions="Select the team responsible for resolving this issue.",
            criteria={
                "core_engineering": "Database timeouts, SDK bugs, or runtime exceptions",
                "customer_success": "General onboarding, billing questions, or feature requests",
                "security": "Breaches, credential compromise, or vulnerability disclosures"
            }
        ),
        # Ordinal urgency
        "severity": Score(
            instructions="Grade operational impact based on revenue disruption and service degradation.",
            levels=[
                "p3_low",       # Minor bug, no operational impact
                "p2_medium",    # Degraded feature, workaround exists
                "p1_critical"   # Production checkout outage, active revenue loss
            ]
        ),
        # Binary guardrail
        "page_oncall": Noul(
            instructions="Based on 'customer_tier' and operational impact, does this incident require paging the on-call engineer immediately?"
        )
    }
)

# Read results via response.answers
target = response.answers["routing_target"]
sev = response.answers["severity"]
page = response.answers["page_oncall"]

print(f"Route: {target.choice} ({target.confidence:.1%})")
print(f"Severity: {sev.score:.2f} (Confidence: {sev.confidence:.1%})")
print(f"Page On-Call Probability: {page.noul:.1%}")

Related Context

Structuring State for TypeSafe Jev

workflow↗

Jev Decision Primitives: Choice vs. Score vs. Noul

comparison↗

Understanding Confidence Scores and Calibrated Probabilities

concept↗