Handle events asynchronously
Configure your handler to process incoming events with an asynchronous queue. Processing transactions synchronously can lead to long response times and potential timeouts, particularly during periods of high market volatility when transaction volumes surge.
If all events are handled synchronously, sudden surges in activity could overwhelm your system and degrade performance. Asynchronous queues allow you to manage these transaction bursts efficiently, ensuring your system processes events at a sustainable rate without bottlenecks or failures.
Respond with a 2xx status promptly
Your endpoint should return a successful 2xx status code as soon as possible, before executing any time-consuming logic that might lead to a timeout. For instance, send a 200 response before processing tasks like marking a customer’s invoice as paid in your accounting system.
Managing duplicate events
Your webhook endpoint may sometimes receive the same event multiple times. To prevent duplicate processing, track event IDs and ignore any that have already been handled.
Additionally, ensure you're processing the most recent event by comparing timestamps in the createdAt field, rather than relying solely on the event ID.
Event ordering
Although Fireblocks strives to send notifications in order per resource, we cannot guarantee that events will always be delivered in the sequence they are generated. We recommend designing your endpoint to process events independently, without relying on their delivery order.
Circuit breaker
Fireblocks introduces a circuit breaker mechanism for managing faulty webhook endpoints to ensure fairness and high throughput across all users. This feature helps identify and suspend endpoints that consistently fail, ensuring that system resources are optimized and reliable for everyone.
The circuit breaker monitors error rates for each webhook endpoint and applies suspension policies based on predefined thresholds. These policies vary depending on the type and severity of the errors. Webhooks with an error rate exceeding a specific threshold will be automatically disabled to protect system integrity.
To prevent performance issues in your application, handle events asynchronously with queues and quickly return a 2xx status code.
Recovering from downtime
If your webhook server was offline for an hour or longer due to a deployment or network issue, you can resend all missed or failed webhook notifications for a given webhook. You can set the resend duration for up to 24 hours.