Dotnet | The AWS Blog

Beyond the SDK Demo: Building Production-Ready .NET Agents with AgentCore

Emiliano Montesdeoca — Fri, 07 Aug 2026 00:00:00 +0000

The first .NET AI agent is usually easy to demonstrate. Register a model client, add a handler, send a prompt, print the answer. The production version is where the interesting decisions begin: how do sessions stay separate, how do responses stream, how do you test without AWS credentials, and what exactly happens when the runtime scales or restarts?

The AWS Developer Tools Blog’s AgentCore guide for .NET introduces AWS.AgentCore.Hosting and the surrounding tooling for those concerns. The library “handles the operational concerns: scaling, session routing, health checking, and providing managed capabilities like conversation memory.” That is useful, but a hosting library cannot choose the boundaries of your application for you.

Choose an API shape that leaves room

The package supports a source-generator experience and an extension-method experience in ASP.NET Core. The choice looks like syntax. It is really a choice about how much control the application will need around the handler.

If the agent is a small, stable endpoint, generated wiring can keep the code compact. If the agent will need custom middleware, request correlation, policy checks, metrics, or dependency injection behavior, the extension method gives the team more room to shape the host. The source article describes middleware that can intercept every invocation. That is exactly where I would put cross-cutting concerns rather than scattering them through prompt handlers.

Keep the handler thin either way. Put business operations, model selection, and external calls behind services that can be tested without the AgentCore runtime. Then the hosting layer remains an integration boundary, not the place where the entire application lives.

Streaming changes the contract

Returning a complete string is comfortable, but it makes users wait for the slowest part of the generation. AgentCore supports returning IAsyncEnumerable<string> so tokens can be streamed as they are produced. For conversational applications, the difference between first token latency and total response latency is visible immediately.

Streaming also changes failure handling. A model call, memory lookup, or downstream tool can fail after the response has started. The caller cannot receive a new HTTP status code at that point, so the protocol needs a clear way to represent an error in the stream. Cancellation matters as well: when the browser disconnects, the handler should stop generating and release work instead of continuing to pay for a response nobody will read.

Test partial output, cancellation, slow clients, and a failure after the first token. A streaming demo proves that bytes arrive. A production test proves that the system behaves when the stream is interrupted.

Session IDs are security boundaries

AgentCore Memory combines a configured Memory ID with a session ID to load and store conversation history. That makes the session ID part of your security model, not just a convenience parameter.

Two users must never share a session by accident. A missing session ID should fail clearly or create a deliberately scoped new session; it should not quietly fall back to a shared default. Add tests for two independent sessions, repeated calls in the same session, missing identifiers, and attempts to reuse another user’s identifier.

The local emulator and AWS.AgentCore.Testing package are valuable here. Use them to verify session behavior in CI without deploying the agent or requiring live AWS credentials. Memory integration is exactly the kind of feature that appears correct in a happy-path demo and fails in a multi-user system.

Deployment is still architecture

The dotnet aws deploy workflow makes deployment approachable, but it also hides choices. Container architecture, memory, VPC access, request header allowlists, and private service connectivity all affect runtime behavior.

Read the generated resources and make those choices explicit. If the agent needs a private database or internal API, test the VPC path from the deployed runtime rather than assuming the local Aspire application proves it. If the service has a meaningful cold-start budget, measure it on the actual architecture you plan to run.

Native AOT can reduce startup time, but it changes the development experience. Source-generated serialization and explicit service resolution may be required where flexible reflection-based binding previously worked. Decide whether cold-start reduction is worth that constraint for this workload instead of enabling AOT because it is available.

Keep portability in the business layer

AgentCore is a managed runtime, but the agent’s business logic can remain portable. Treat the handler as an adapter around services that know how to perform domain operations. Avoid spreading AgentCore-specific session and transport calls through every class.

That design gives you a useful fallback: the same domain service can be hosted in another ASP.NET Core application, tested in isolation, or moved to another container platform if the operational requirement changes. Portability is not free, but it is much cheaper when the boundary is intentional from the beginning.

My pre-production checklist

Before the first real users arrive, I would require:

separate-session tests and a clear session ID ownership model,
streaming tests covering cancellation and mid-response failures,
emulator-backed integration tests in CI,
health and correlation telemetry around every invocation,
a documented deployment and rollback path,
a decision on JIT versus Native AOT based on measured startup data.

The AgentCore hosting library removes a lot of plumbing. That is its value. The remaining work is the architecture around the plumbing: state, failure, identity, and operations. The SDK demo gets you to the first response. Those decisions are what get the agent to production.

Cross-Account ECS Telemetry Needs a Platform Boundary

Emiliano Montesdeoca — Fri, 07 Aug 2026 00:00:00 +0000

A collector sidecar is a friendly pattern right up until an organization has hundreds of ECS tasks. Then the same convenience becomes a platform tax: every task carries collector CPU and memory, every account carries configuration, and every update becomes another rollout.

The problem is even sharper for Windows .NET Framework workloads. A Linux collector sidecar is not a universal answer for an IIS application running on Windows. You either create a separate collection strategy or accept a hole in the telemetry map.

The AWS Containers Blog’s cross-account ECS telemetry pattern moves collection into a dedicated observability account. Workloads send OpenTelemetry Protocol data over private connectivity to a centralized AWS Distro for OpenTelemetry gateway, which exports traces to AWS X-Ray and metrics and logs to CloudWatch.

That is more than a collector deployment. It is an observability platform boundary.

Sidecar versus gateway

The sidecar model has a real advantage: the collector sits next to the application and is easy to understand. For a small service, that local ownership may be exactly right. The cost appears when the pattern is copied everywhere.

A collector per task multiplies baseline resource usage. It also multiplies configuration drift. A change to batching, sampling, authentication, or metric dimensions must reach every task definition and every account. Windows workloads introduce another constraint because the application may not be able to host the same Linux sidecar at all.

A centralized gateway reverses those trade-offs:

one fleet to patch and configure,
one endpoint for workloads across accounts,
one place to control batching and export behavior,
a shared path for Linux and Windows applications.

The price is network and platform complexity. You need private routing between workload VPCs and the observability VPC, capacity planning for the gateway, and a clear ownership model for telemetry arriving from many accounts.

Windows .NET is the forcing function

For a Windows .NET Framework application hosted in IIS, the gateway is not just a cost optimization. It can be the practical way to reach the destination telemetry systems without running a Linux collector beside the application.

The source pattern configures OpenTelemetry instrumentation in the Windows image and points the OTLP exporter at the gateway’s internal network load balancer. IIS worker processes need the right machine-level environment variables and instrumentation registration. That detail is easy to miss because a Linux container tutorial can make environment configuration look universal.

The useful design principle is to keep application instrumentation consistent while moving platform-specific collection into the shared gateway. The application emits standard OTLP. The gateway owns export, batching, dimensions, and destination-specific details.

The failure mode that wastes a day

Centralization does not make networking disappear. One common failure is a target health-check loop: ECS tasks send telemetry to the load balancer, but the collector targets never become healthy. The security group allows client CIDRs but blocks health checks originating from the load balancer subnets.

Allow the health-check path explicitly. Test the gateway’s health endpoint from the load balancer perspective, then test OTLP reachability from a workload VPC. A collector can be running and still be unreachable, which makes task logs unnecessarily confusing.

Preserve source identity as well. The gateway should add fallback metadata only when the workload did not provide it. If the collector overwrites account, cluster, or service attributes, all telemetry starts looking as if it came from the observability account. Centralization is useful only when the origin remains trustworthy.

Choose the layer that is missing

The AWS article distinguishes collection from visualization. CloudWatch cross-account observability can give operators a shared view of telemetry that is already being collected. It does not solve an instrumentation or ingestion gap. An ADOT gateway operates earlier, at the collection boundary.

That distinction keeps teams from deploying a gateway to solve the wrong problem. If every workload already emits reliable telemetry and the pain is searching across accounts, use the native cross-account viewing features. If applications cannot collect consistently, or you need one governed exporter configuration, the gateway is the better fit.

A practical rollout

I would start with one observability account, two Availability Zones, and one representative Linux service plus one Windows .NET service. Before onboarding more accounts:

Establish Transit Gateway or equivalent private routing and verify non-overlapping CIDR ranges.
Run the gateway behind an internal load balancer and alarm on unhealthy targets.
Send a test trace, metric, and log from each operating-system family.
Check that account, cluster, service, and environment dimensions survive the gateway.
Limit CloudWatch metric dimensions to fields you actually query; high-cardinality metadata becomes an ingestion bill quickly.
Add a second collector task and exercise a replacement while telemetry is flowing.

The source article estimates that a small centralized gateway can run in the low tens of dollars per month before telemetry ingestion costs. The exact number will vary, but the broader lesson is stable: compare one governed fleet with the cumulative cost of per-task collectors, not with a zero-cost imaginary sidecar.

A centralized ADOT gateway is not automatically the right answer for every ECS estate. It is a strong answer when multi-account scale, Windows .NET support, and consistent collection are the actual constraints. Treat it as platform infrastructure, preserve source identity, and test the failure path before calling observability complete.

Response Streaming for .NET on Lambda Changes Perceived Latency

Emiliano Montesdeoca — Fri, 07 Aug 2026 00:00:00 +0000

A response that takes five seconds can feel faster than a response that takes three seconds if the first one starts showing useful content immediately. That is the practical reason response streaming matters for .NET workloads on AWS Lambda.

The AWS announcement adds response streaming support for .NET Lambda functions. Instead of buffering the complete result and returning it once the handler finishes, the function can send bytes as they become available. That is especially useful for token generation, large exports, and responses that do not fit comfortably in the traditional buffer.

The feature is valuable. It also changes the failure model of the endpoint.

Buffering hides time from the user

A model can generate a token at a time, but a buffered Lambda response makes the client wait for all of them. The same problem appears with a large CSV export: the function spends time building the response, and the caller sees nothing until serialization and buffering are complete.

Streaming changes that first-byte experience. With a StreamWriter, the handler writes a chunk and flushes it to the caller. For an LLM, that means the interface can render the first tokens while the model continues working.

The flush is important. A streaming API that never flushes is just a buffered API with extra ceremony. The source examples use explicit FlushAsync calls, and your implementation should choose a flush strategy based on the client and payload rather than assuming every write crosses the network immediately.

Headers are a one-time decision

The API Gateway integration adds a constraint that is easy to miss: the response needs a prelude containing the HTTP status code and headers. Once the prelude has gone out, the handler cannot change the status code because a later exception occurs.

That means validation and setup should happen before the first response bytes. If a model call can fail, decide how the stream will represent that failure after headers have been sent. A JSON error body cannot magically turn a previously sent 200 into a 500.

The API Gateway integration also needs the response-streaming invocation path and the correct transfer mode. If the function is configured for streaming but the gateway still buffers, the client receives none of the latency benefit. Test the deployed path, not only the handler in isolation.

Timeouts and disconnects become visible

Streaming does not reset the Lambda timeout. A function with a 30-second timeout still has a 30-second budget, even if it flushes a token every second. API Gateway also needs its own timeout configuration and enough headroom for the integration to finish.

Client disconnects create a second operational concern. If the browser closes after token 40, Lambda may continue generating token 41 through 200 unless the handler observes cancellation or a failed flush. That work costs money and may keep a model invocation alive after the user has gone away.

Log the session or request ID, time to first byte, number of chunks sent, total duration, and the reason the stream ended. A normal completion, a client disconnect, and a model failure should not look identical in CloudWatch.

ASP.NET Core is a useful path

For ASP.NET Core applications hosted on Lambda, the AWS hosting layer can enable response streaming with EnableResponseStreaming. Standard response helpers can continue to work, but the integration still needs to be configured and tested.

A minimal endpoint might write a line, flush periodically, and stop when the request is cancelled. Keep the stream format stable. For token output, newline-delimited events or a documented event format is easier for clients to process than arbitrary text fragments. For large exports, make the client aware that a partial response is not a valid completed file.

Where the feature fits

The larger response ceiling is useful, but it should not become an excuse to send unbounded data through a synchronous Lambda request. A 200 MB limit does not make a long-lived download a good fit for every API. Use S3 for durable files and presigned delivery when that is the simpler contract.

I would prioritize streaming for interactive AI responses and incremental exports where first-byte latency matters. For normal CRUD endpoints, buffering remains easier to observe and retry.

What to test before production

Test the full path with:

a slow model or generator,
a client that disconnects halfway through,
an exception after the first flush,
a response that approaches the integration limit,
API Gateway timeout headroom,
retries after a partial response.

Also measure perceived latency rather than only total duration. The point of streaming is not to make the model compute less. It is to deliver useful output sooner without turning partial output into an ambiguous API.

For .NET teams building serverless AI features, response streaming is a meaningful improvement. Just remember that once bytes leave the function, the response contract is already in motion. Headers, cancellation, observability, and client behavior need to be designed along with the IAsyncEnumerable or stream itself.