Performance

Designing high-throughput background pipelines with C# System.Threading.Channels When our project receives thousands of requests per minute, we need to take some cares. Usually, the bottleneck isn’t CPU. Probabily the problem is one (or more) of these: I/O contention, thread starvation, uncontrolled concurrency, excessive allocations… et cetera. One of the most common mistakes in high-throughput (even in medium-throughput) APIs is coupling request processing with expensive secondary operations such as: audit logging; telemetry; webhook dispatching; email sending; analytics; event persistence. So it causes response times increasing, thread pressure growing and throughput collapses. ...

I recently published a detailed technical article on Balta.io-one of the premier .NET ecosystem references in Brazil-addressing a critical topic for high-performance systems: concurrency control. As software engineers with extensive experience in high-demand solutions, we often face the challenge of “doing as much as possible, as fast as possible”. However, infrastructure realities-such as rate limits in third-party APIs or hardware constraints-force us to understand exactly when and how to throttle our execution. ...

If you’re trying to insert millions of rows using INSERT in a loop, you’re doing it wrong. A few days ago, I was asked about it. It’s funny how a tool you use usually can slip our mind. Anyway, BULK INSERT is one of the most efficient ways to load large datasets into SQL Server. It can increase the insertion speed by up to 10x to 50x, which means that it can significantly reduce the time required for the operation. ...

Have you worked with Split Queries in EF Core before? When you incorporate related tables using EF Core, it typically generates a JOIN query, potentially leading to duplicated data. While this duplication is usually insignificant for small datasets, it can become problematic when dealing with larger volumes. When multiple collections are included, SQL JOINs multiply rows exponentially, not linearly. 1 2 3 4 var posts = context.Posts .Include(p => p.Comments) .Include(p => p.Tags) .ToList(); Imagine a scenario where you have a table with 1000 posts and each post has 1000 comments. If you use the query below, you will get 1000 * 1000 = 1,000,000 rows in the result. This phenomenon is known as Cartesian Explosion, causing unexpected spikes in data loads. ...