GenServer Patterns for Stateful Services

I'm building a GenServer that manages a pool of database connections. What patterns should I use to handle connection failures and automatic reconnection?

For a connection pool GenServer, I'd recommend a supervision tree approach combined with the circuit breaker pattern: 1. **Supervisor + GenServer per connection**: Each connection gets its own GenServer under a DynamicSupervisor. When a connection dies, the supervisor restarts it automatically. 2. **Health check via `handle_info`**: Use `Process.send_after(self(), :health_check, 30_000)` to periodically verify connections. On failure, transition to a `:reconnecting` state. 3. **Circuit breaker**: Track consecutive failures. After N failures, stop attempting for a backoff period. Use exponential backoff: `min(30_000, 1_000 * :math.pow(2, attempt))`. 4. **ETS for fast reads**: Keep the pool state in an ETS table so consumers can check availability without messaging the GenServer.

How should I handle the case where a consumer is already using a connection and it drops mid-query?

The key insight is: **the consumer should never hold a raw connection reference**. Instead, use a checkout/checkin pattern with monitors: 1. **Checkout returns a reference, not the connection**: The pool GenServer monitors the consumer process. If the consumer dies, the connection is automatically returned. 2. **Wrap the operation**: Instead of giving the connection out, accept a function and run it inside the GenServer. 3. **Timeout with `Task.async` for safety**: If the query hangs, wrap in a Task with a timeout. The GenServer should never block indefinitely. This pattern is exactly what `db_connection` (the library behind Ecto's pool) uses internally.

Good call on db_connection. What about for non-database resources — like managing a pool of HTTP client connections?

For HTTP connection pools, the pattern is simpler because HTTP is inherently stateless per-request. Use `Finch` (the HTTP client behind `Req`) which already manages connection pools via NimblePool. For custom resource pools (WebSocket connections, gRPC channels, external service sessions), NimblePool is the building block. The key design principle: **let the pool library handle lifecycle, you just define init/checkout/checkin/terminate**. Don't roll your own pool management unless you have very unusual constraints.