A model request is a lifecycle, not a string function
How to study this lesson
Read one section, trace the worked example, and try the knowledge check. Bookmark saves a shortcut; notes appear in My notebook. Mark lesson complete is your own assessment and does not mark its lab passed.
By the end, you can
- Build a documented Responses API request with environment configuration.
- Distinguish streamed text from completed output.
- Record request evidence and classify integration failures.
Separate application policy from provider transport
A useful integration has a narrow application interface such as generate_answer(request) returning a typed result. The provider adapter translates that request into the chosen API and translates its lifecycle back into your application's statuses. This separation lets you test state handling with recorded fixtures while reserving live requests for checking model behavior and integration compatibility.
The example uses the OpenAI Python SDK's Responses interface, checked against official documentation on 2026-09-07. Install a compatible openai package, configure OPENAI_API_KEY on the trusted backend, and set OPENAI_MODEL to a model identifier available to your account with the required features. The code intentionally has no invented default model. Record the installed SDK version and resolved model identifier with experiments so a later reader can reproduce the request as closely as the service permits.
Define the request before tuning generation
A request contains more than a user sentence. It includes application instructions, input content, model selection, output constraints, and possibly tool definitions or conversation state. Keep durable application rules distinct from the user's data. Do not concatenate credentials or hidden operational context into a prompt merely because it is convenient for debugging.
Start with a small request whose success criteria can be checked. For example, ask for a two-sentence explanation of a cache miss, then verify that your application receives a completed textual answer and preserves the associated status. Once the transport path is sound, improve the prompt or model selection using evaluation examples. Changing infrastructure, prompt, and sampling settings simultaneously makes a failure harder to attribute.
Streaming changes delivery, not truth
Streaming lets the application receive output incrementally. In the documented Responses stream, text deltas arrive as typed events, and lifecycle events indicate completion or failure. A delta is a fragment to append to the appropriate output channel; it is not a complete answer or necessarily a valid JSON document. The application must keep incomplete content visibly distinct from a final result.
Imagine receiving the fragments "The total is " and "12" before the connection drops. The screen can show those characters, but the run has not established a completed answer. A UI that changes to done whenever any text appears will misrepresent this case. Preserve both accumulated text and terminal status, and use an explicit incomplete state when the stream ends without a recognized successful terminal event.
Failures have different retry implications
A validation error, an authentication failure, a rate limit, a transport interruption, and a model refusal require different handling. Retrying malformed input without changing it wastes time. Retrying after a transport failure may be reasonable for a read-only generation request, but it can still increase cost. If a request participates in a tool workflow with external side effects, replay must be governed by the action's idempotency policy.
Set a total request deadline and a retry budget at the application level. Respect documented provider behavior and avoid stacking your own retries on top of SDK retries without understanding the combined maximum. Log a safe request identifier, status, latency, and available usage metadata. Do not log the API key or assume every raw prompt is appropriate for unrestricted operational logs.
Test the boundary before measuring the model
The code appends text deltas, recognizes successful completion, and rejects a stream that ends without it. Its output is intentionally labeled illustrative because it depends on the selected model and live service; no request was made during course validation. The offline lab later tests the same lifecycle idea with normalized application events, including missing completion and events after termination.
Separate two questions in your test plan. Does the adapter obey the protocol and report failures accurately? Does the model produce useful answers for representative requests? Fixtures answer the first deterministically, while a controlled evaluation answers the second with statistical uncertainty. A passing parser test cannot establish answer quality, and an excellent demo answer cannot establish correct timeout or cancellation behavior.
Work through the code
Prerequisites: a compatible openai SDK, server-side OPENAI_API_KEY, OPENAI_MODEL set to an available streaming-capable model, and network access. The snippet uses documented Responses events and makes a live request when run. It was syntax-compiled but not executed against the API during course validation. For multiple output items, route deltas by output/content identity instead of using this single-text display buffer.
import os
from openai import OpenAI
client = OpenAI()
model = os.environ["OPENAI_MODEL"]
stream = client.responses.create(
model=model,
instructions="Explain the requested software concept in two sentences.",
input="What is a cache miss?",
stream=True,
)
parts = []
completed = False
for event in stream:
if event.type == "response.output_text.delta":
parts.append(event.delta)
print(event.delta, end="", flush=True)
elif event.type == "response.completed":
completed = True
elif event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(f"Generation did not complete: {event.type}")
if not completed:
raise RuntimeError("Stream ended without completion")
print("\nStatus: completed")
Illustrative output: depends on the configured model and account; no API request was made during course validation.
Run Python snippets locally with the prerequisites named above. The module coding lab runs directly in your browser.
Pause and reason
A stream sends three text deltas and then the socket closes without a completion event. What should your API response and UI state retain, and what should they avoid claiming?
Check your understanding
What does receiving the first streamed text delta prove?
Your notes
Explain the mechanism in your own words. Add a failure you want to test.