REST APIs, JSON and OAuth 2.0 authentication

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Concept: API design · also covers REST APIs, JSON · Step 10 of 34 in AI forward deployed engineer · version 1

Practice this

On this page

Most enterprise integrations are HTTP calls carrying JSON, authorized by an OAuth 2.0 access token. The job of a forward deployed engineer is less about designing a clean API and more about consuming someone else's imperfect one safely: getting delegated access approved, validating payloads, surviving rate limits and writing back to a system of record without creating duplicates. This lesson moves from the protocol layer, through data contracts, to authorization and resilient client design.

Read more

How it is done in practice

A team starts in the vendor sandbox with a collection of saved requests, gets a connected app or service account approved by the customer's admin, and confirms which scopes the security team will accept. They generate or hand-write a thin typed client, put schema validation and structured logging at the boundary, and wrap it with one shared policy for timeouts, retries, backoff and token refresh. Read sync ships first with a watermark and a reconciliation count against the source; writeback ships second, behind an upsert key and a dry-run mode that logs intended changes, and is enabled for a handful of records before it is opened to the whole tenant.

Technologies

OpenAPIJSON SchemaOAuth 2.0PKCEPostmancurlPython requestshttpxPydanticZodOktaMicrosoft Entra ID

Diagram

sequenceDiagram
  participant U as User
  participant C as Client app
  participant A as Authorization server
  participant R as Resource server
  U->>C: Connect account
  C->>A: Authorization request with state and PKCE challenge
  A->>U: Login and consent
  A->>C: Redirect with authorization code
  C->>A: Exchange code and verifier
  A->>C: Access token and refresh token
  C->>R: API call with Bearer token
  R->>C: JSON response
sequenceDiagram
  participant U as User
  participant C as Client app
  participant A as Authorization server
  participant R as Resource server
  U->>C: Connect account
  C->>A: Authorization request with state and PKCE challenge
  A->>U: Login and consent
  A->>C: Redirect with authorization code
  C->>A: Exchange code and verifier
  A->>C: Access token and refresh token
  C->>R: API call with Bearer token
  R->>C: JSON response

Code

python
import os, requests

def get_token():
    r = requests.post(os.environ["TOKEN_URL"], data={
        "grant_type": "client_credentials",
        "scope": "contacts.read",
    }, auth=(os.environ["CLIENT_ID"], os.environ["CLIENT_SECRET"]), timeout=10)
    r.raise_for_status()
    return r.json()["access_token"]

def list_contacts(token):
    url, out = os.environ["API_BASE"] + "/contacts", []
    while url:
        r = requests.get(url, headers={"Authorization": f"Bearer {token}"}, timeout=10)
        r.raise_for_status()
        body = r.json()
        out += body["items"]
        url = body.get("next")  # cursor link, absent on last page
    return out

Subtopics

HTTP semantics from the consumer side

What matters when you consume instead of serve

HTTP is a stateless client-server protocol: every request carries a method, a path, headers and an optional body, and every response carries a status code. As a consumer, the method semantics decide your retry policy. GET, PUT and DELETE are defined as idempotent, so a timed-out call can be repeated; POST is not, so a blind retry can create a second record. Status classes decide the next action: 4xx means fix the request or the credentials, 429 means slow down, 5xx means retry later. Vendor APIs often bend these rules, so confirm actual behavior in a sandbox before trusting the documentation.

JSON, JSON Schema and OpenAPI as the contract

JSON and its sharp edges

JSON has six value kinds: object, array, string, number, true or false, and null. The format does not define integer width, dates, or what to do with duplicate keys, and that is where integrations break. Large identifiers lose precision when a parser reads them as floating point, so many APIs send them as strings. Timestamps arrive as strings in whatever format the vendor chose. Missing key, null and empty string are three different things that source systems use inconsistently. Decide explicitly how each maps into your own model instead of letting the parser decide.

Schemas at the boundary

JSON Schema describes the shape of a document: types, required properties, enums, formats and nested structures. Validating every inbound payload against a schema at the boundary turns silent data corruption into a loud, attributable error. OpenAPI wraps the same idea around a whole HTTP API: paths, operations, parameters, request and response schemas, and security schemes such as OAuth 2.0 flows and scopes. When a vendor publishes an OpenAPI document, generate a typed client from it and diff new versions in CI. When they do not, write the schema yourself from observed traffic and treat it as your contract test.

OAuth 2.0 roles and grant types

The four roles

OAuth 2.0 is an authorization framework that lets an application obtain limited access to an HTTP service without holding the user's password. It defines four roles. The resource owner is the user or organization that owns the data. The client is your application. The authorization server authenticates the owner, collects consent and issues tokens. The resource server is the API that accepts those tokens. In enterprise SaaS the authorization server and resource server are usually the same vendor, and the customer's admin, not the end user, often has to approve your client before anyone can connect.

Choosing a grant

The authorization code grant is for acting on behalf of a user: the browser is redirected to the authorization server, the user consents, and your backend exchanges the returned code for tokens. Add PKCE and a state parameter to protect the exchange. The client credentials grant is for machine to machine access where the client acts as itself, which is typical for backend sync jobs and service accounts. The original specification also defines the implicit and resource owner password grants; current guidance treats both as legacy and they should not be used for new work. A refresh token lets the client obtain new access tokens without user interaction.

Token lifecycle, scopes and secret storage

Running tokens in production

Access tokens are short lived by design, so the client must handle expiry as a normal path. Cache the token with its expiry time, refresh slightly early, and on a 401 refresh once and retry once. Some authorization servers rotate the refresh token on every use, so concurrent workers must serialize the refresh or one of them will be logged out. Request the narrowest scopes that the workflow needs; security reviewers will ask why each one is there. In a multi-tenant deployment, store tokens per customer tenant, encrypted, in a secrets manager or an encrypted column, never in logs, prompts or agent context.

What an AI agent changes

When an LLM agent calls APIs through tools, the token defines the blast radius of a bad decision. Keep the token inside the tool implementation so the model never sees it. Prefer per-user delegated tokens over one powerful service account, so the agent can only do what the requesting user could do, and the vendor's audit log shows who acted. Split read scopes from write scopes and gate write tools behind approval where the action is hard to undo.

Resilient clients: pagination, rate limits and safe writeback

Reading at volume

Bulk reads go through pagination. Cursor based pagination returns an opaque pointer to the next page and stays stable while data changes; offset pagination can skip or repeat rows when records are inserted mid-scan. For incremental sync, filter on a last-modified field and keep a watermark, overlapping slightly and deduplicating by id. Respect rate limits: read the limit headers the vendor returns, honor Retry-After on 429, and back off exponentially with jitter. Budget the limit across all your workers for a tenant, since the quota is usually shared with the customer's other integrations.

Writing without duplicates

Writeback to a CRM or ERP is where mistakes become visible to the customer. A timeout does not tell you whether the write happened. Use an idempotency key header where the API supports it, or an upsert on an external id field that you control, so repeating the call is harmless. Otherwise read before write. Record every outbound mutation with request id, tenant, payload hash and response, so a wrong update can be traced and reversed.

Sources

Related lessons

Practice this
Shipping production code with Claude Code and other AI coding toolsWebhooks and event-driven integrations with Kafka