Skip to content
Skills Library
Programming · Building & coding

optimizing-io-operations

/optimizing-io-operations

Optimizes standard I/O and file operations for high-performance data processing in .NET. Use when building high-throughput file processing or competitive programming solutions.

Licence
MIT
Last repository change
21.07.2026

Assessment and security

84/ 100Class: good
Format9 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content35 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
2
Scripts
0
HTML comments
0
URLs in SKILL.md
2
Deprecation

No signals detected. Repository changed on 21.07.2026, without references to retired models or APIs.

Licence

MIT, open: use and modify while retaining author attribution.

What it does and when to use it

Optimizes standard I/O and file operations for high-performance data processing in .NET. Use when building high-throughput file processing or competitive programming solutions.

Original author description

Optimizes standard I/O and file operations for high-performance data processing in .NET. Use when building high-throughput file processing or competitive programming solutions.

How it works

  1. 1. Core APIs
  2. 2. High-Speed Standard I/O
  3. 3. File I/O Optimization
  4. 4. Large File Processing
  5. 5. Performance Comparison
  6. 6. Important Notes
  7. 7. References
Original SKILL.md headings · 233 lines, 601 words
SKILL.md preview
---
description: "Optimizes standard I/O and file operations for high-performance data processing in .NET. Use when building high-throughput file processing or competitive programming solutions."
user-invocable: false
model: sonnet
---

# .NET High-Performance I/O

A guide for APIs optimizing large-scale data input/output.

**Quick Reference:** See [QUICKREF.md](QUICKREF.md) for essential patterns at a glance.

## 1. Core APIs

| API | Purpose |
|-----|---------|
| `Console.OpenStandardInput()` | Buffered stream input |
| `Console.OpenStandardOutput()` | Buffered stream output |
| `BufferedStream` | Stream buffering |
| `FileOptions.Asynchronous` | Async file I/O |

---

## 2. High-Speed Standard I/O

### 2.1 Basic Pattern

```csharp
// Use buffer stream directly for large I/O
using var inputStream = Console.OpenStandardInput();
using var outputStream = Console.OpenStandardOutput();
using var reader = new StreamReader(inputStream, bufferSize: 65536);
using var writer = new StreamWriter(outputStream, bufferSize: 65536);

// Disable buffer flush for performance improvement
writer.AutoFlush = false;

string? line;
while ((line = reader.ReadLine()) is not null)
{
    writer.WriteLine(ProcessLine(line));
}

// Manual flush at the end
writer.Flush();
```

### 2.2 For Algorithm Problem Solving

```csharp
using System.Text;

// High-speed input
using var reader = new StreamReader(
    Console.OpenStandardInput(),
    Encoding.ASCII,
    bufferSize: 65536);

// High-speed output
using var writer = new StreamWriter(
    Console.OpenStandardOutput(),
    Encoding.ASCII,
    bufferSize: 65536);

var sb = new StringBuilder();

// Collect large output in StringBuilder and write at once
for (int i = 0; i < 100000; i++)
{
    sb.AppendLine(i.ToString());
}

writer.Write(sb);
writer.Flush();
```

---

## 3. File I/O Optimization

### 3.1 Buffer Size Optimization

```csharp
// Use larger buffer than default (4KB)
const int bufferSize = 64 * 1024; // 64KB

using var fileStream = new FileStream(
    path,
    FileMode.Open,
    FileAccess.Read,
    FileShare.Read,
    bufferSize: bufferSize);
```

### 3.2 Async File I/O

```csharp
// Open file with async option
using var fileStream = new FileStream(
    path,
    FileMode.Open,
    FileAccess.Read,
    FileShare.Read,
    bufferSize: 4096,
    options: FileOptions.Asynchronous);

var buffer = new byte[4096];
int bytesRead = await fileStream.ReadAsync(buffer);
```

### 3.3 SequentialScan Hint

```csharp
// Provide hint to OS for sequential reading
using var fileStream = new FileStream(
    path,
    FileMode.Open,
    FileAccess.Read,
    FileShare.Read,
    bufferSize: 64 * 1024,
    options: FileOptions.SequentialScan);
```

### 3.4 RandomAccess (.NET 6+)

```csharp
// Direct offset access without file position management
using var handle = File.OpenHandle(path, FileMode.Open, FileAccess.Read);

var buffer = new byte[4096];
long offset = 1000;

int bytesRead = RandomAccess.Read(handle, buffer, offset);

// Async version
bytesRead = await RandomAccess.ReadAsync(handle, buffer, offset);
```

---

## 4. Large File Processing

### 4.1 Chunk-Based Reading

```csharp
public async IAsyncEnumerable<byte[]> ReadChunksAsync(
    string path,
    int chunkSize = 64 * 1024,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    using var stream = new FileStream(
        path,
        FileMode.Open,
        FileAccess.Read,
        FileShare.Read,
        bufferSize: chunkSize,
        options: FileOptions.Asynchronous | FileOptions.SequentialScan);

    var buffer = new byte[chunkSize];
    int bytesRead;

    while ((bytesRead = await stream.ReadAsync(buffer, ct)) > 0)
    {
        if (bytesRead == chunkSize)
        {
            yield return buffer;
            buffer = new byte[chunkSize];
        }
        else
        {
            yield return buffer[..bytesRead];
        }
    }
}
```

### 4.2 Memory-Mapped Files

```csharp
using System.IO.MemoryMappedFiles;

// Map large file to memory
using var mmf = MemoryMappedFile.CreateFromFile(path, FileMode.Open);
using var accessor = mmf.CreateViewAccessor();

// Direct memory access
byte value = accessor.ReadByte(position);
accessor.Write(position, newValue);
```

---

## 5. Performance Comparison

| Method | Relative Performance | Use Case |
|--------|---------------------|----------|
| Console.ReadLine() | 1x (baseline) | General |
| StreamReader (default buffer) | 2x | Large data |
| StreamReader (64KB buffer) | 3-5x | Large data |
| MemoryMappedFile | 5-10x | Very large data |

---

## 6. Important Notes

### Buffer Size

- Too small increases system calls
- Too large wastes memory
- Recommended: 4KB ~ 64KB

### Encoding Specification

```csharp
// Read UTF-8 without BOM
using var reader = new StreamReader(
    stream,
    new UTF8Encoding(encoderShouldEmitUTF8Identifier: false));
```

### Flush Timing

```csharp
// Improve performance with AutoFlush = false
writer.AutoFlush = false;

// Manual flush after important data
writer.Flush();
```

---

## 7. References

- [File and Stream I/O](https://learn.microsoft.com/en-us/dotnet/standard/io/)
- [Memory-Mapped Files](https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files)

Files in this package (2)
  • QUICKREF.md 1 KB
  • SKILL.md 5 KB

Versions

The database has no other skills named optimizing-io-operations.

Database snapshot: 13.11.2026 · package version 17c8b9b1221e (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 90file-processing
    Stream processing, large file uploads, and IO efficiency. Use when: file processing.
  • 84implementing-io-pipelines
    Implements high-performance streaming using System.IO.Pipelines in .NET. Use when building network protocols, parsing binary data, or processing large streams efficiently.
  • 84optimizing-memory-allocation
    Implements Zero Allocation patterns using Span, ArrayPool, and ObjectPool for memory efficiency in .NET. Use when reducing GC pressure or optimizing high-performance memory operations.
  • 100optimizing-performance
    Analyzes and optimizes application performance across frontend, backend, and database layers. Use when diagnosing slowness, improving load times, optimizing queries, reducing bundle size, or when asked about performance issues.
  • 100system-io-pipelines
    Guidance for System.IO.Pipelines high-performance I/O in .NET. USE FOR: high-throughput stream parsing, zero-copy buffer management, PipeReader/PipeWriter patterns, network protocol parsing, ReadOnlySequence processing, replacing Stream-based I/O bottlenecks. DO NOT USE FOR:…
  • 100file-operations
    File operations skill. Use when the user asks to create, read, edit, list, or delete files, or to explore the project directory structure.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.