Przejdź do treści
Biblioteka skilli
Programowanie · Budowanie i kod

file-upload-processor

/file-upload-processor

Buduje przesyłanie plików w aplikacji webowej: punkty końcowe, walidację typu, rozmiaru i magic bytes, integrację z chmurą i śledzenie statusu.

Repozytorium
skills ↗
Licencja
Apache-2.0
Ostatnia zmiana repozytorium
25.10.2026

Ocena i bezpieczeństwo

100/ 100Klasa: dobry
Format25 / 25
Frontmatter z nazwą i opisem, poprawna nazwa, opis do 1024 znaków, nazwa zgodna z folderem.
Odkrywalność25 / 25
Opis mówi, kiedy użyć skilla, ma sensowną długość i jest w 3. osobie.
Treść35 / 35
Co najmniej 150 słów i do 500 linii, nagłówki, listy kroków, przykłady albo kod.
Higiena15 / 15
Bez zaślepek typu TODO, bez ścieżek Windows, z licencją.
Ryzykobrak sygnałów w skanie
Pliki w paczce
2
Skrypty
0
Komentarze HTML
0
Adresy URL w SKILL.md
0
Przestarzałość

Brak sygnałów. Repozytorium zmienione 25.10.2026, bez odwołań do wycofanych modeli i API.

Licencja

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

Co robi i kiedy

Buduje przesyłanie plików w aplikacji webowej: punkty końcowe, walidację typu, rozmiaru i magic bytes, integrację z chmurą i śledzenie statusu.

Wyzwalacze z opisu

file upload,image upload,upload endpoint,multipart upload,presigned URL,S3 upload,file validation,upload to cloud storage,
Oryginalny opis (EN)

When the user needs to build file upload functionality for a web application. Use when the user mentions "file upload," "image upload," "upload endpoint," "multipart upload," "presigned URL," "S3 upload," "file validation," "upload to cloud storage," or "accept user files." Handles upload endpoints, file validation (type, size, magic bytes), cloud storage integration, and upload status tracking. For bucket setup and presigned URLs in depth, see s3-storage.

Jak działa

  1. Overview
  2. Instructions
  3. Examples
  4. Guidelines
Sekcje z nagłówków SKILL.md, w oryginale · 132 linii, 859 słów
Podgląd SKILL.md
---
name: file-upload-processor
description: >-
  When the user needs to build file upload functionality for a web application.
  Use when the user mentions "file upload," "image upload," "upload endpoint,"
  "multipart upload," "presigned URL," "S3 upload," "file validation," "upload
  to cloud storage," or "accept user files." Handles upload endpoints, file
  validation (type, size, magic bytes), cloud storage integration, and upload
  status tracking. For bucket setup and presigned URLs in depth, see s3-storage.
license: Apache-2.0
compatibility: "Node.js 18+, Python 3.9+"
metadata:
  author: terminal-skills
  version: "1.1.0"
  category: development
  tags: "file-upload, cloud-storage, s3, validation"
---

# File Upload Processor

## Overview

Builds secure file upload endpoints for web applications. Handles multipart form uploads, presigned URL generation for large files, file type validation via magic bytes (not just extensions), size limits, cloud storage integration (S3, GCS, R2), and upload status tracking. Buffering matters: multer's memory storage holds each whole file in RAM, so use it only for small files; larger files go to disk or straight to storage.

## Instructions

### 1. Choose Upload Strategy

Based on file size:

- **Small files (< 10MB)**: Stream through server to storage. Simple, one request.
- **Medium files (10-100MB)**: Server-side streaming with progress tracking.
- **Large files (> 100MB)**: Presigned multipart upload — client uploads directly to S3.

### 2. File Validation

Always validate by magic bytes, never trust file extensions:

```typescript
const MAGIC_BYTES = {
  'image/jpeg': [0xFF, 0xD8, 0xFF],
  'image/png': [0x89, 0x50, 0x4E, 0x47],
  'image/webp': [0x52, 0x49, 0x46, 0x46], // + "WEBP" at offset 8
  'application/pdf': [0x25, 0x50, 0x44, 0x46],
  'video/webm': [0x1A, 0x45, 0xDF, 0xA3],
};

function detectFileType(buffer: Buffer): string | null {
  // Read first 12 bytes
  // Match against known signatures
  // Return MIME type or null if unknown
}
```

MP4 has no signature at offset 0. Check that bytes 4–7 spell `ftyp` (`0x66 0x74 0x79 0x70`). WebP needs `RIFF` at offset 0 and `WEBP` at offset 8.

Additional validation:
- Reject early when the `Content-Length` header is above the limit. Chunked requests have no such header, so the parser limit below is still required.
- Set hard limits in the parser. In multer, `limits.fileSize` defaults to `Infinity`, so set it explicitly:

```javascript
const upload = multer({
  storage: multer.memoryStorage(), // small files only, see Guidelines
  limits: { fileSize: 10 * 1024 * 1024, files: 1, fields: 10 },
  fileFilter: (req, file, cb) =>
    cb(null, ['image/jpeg', 'image/png', 'image/webp', 'application/pdf'].includes(file.mimetype)),
});

// Route handler: map multer errors to 413 / 400 instead of a 500
// if (err instanceof multer.MulterError && err.code === 'LIMIT_FILE_SIZE') -> 413
```

- `file.mimetype` comes from the client. Check magic bytes on the buffer after upload.
- Scan for double extensions: `image.jpg.exe`
- Reject files with null bytes in filename

### 3. Storage Integration

```typescript
// S3-compatible storage client
class StorageService {
  async upload(key: string, stream: Readable, contentType: string): Promise<string>
  async getPresignedUploadUrl(key: string, contentType: string, expiresIn: number): Promise<string>
  async getPresignedDownloadUrl(key: string, expiresIn: number): Promise<string>
  async initiateMultipartUpload(key: string): Promise<{ uploadId: string, parts: PresignedPart[] }>
  async completeMultipartUpload(key: string, uploadId: string, parts: CompletedPart[]): Promise<void>
  async delete(key: string): Promise<void>
}
```

To stream a request body to S3 without holding it in memory, pipe the parser into `Upload` from `@aws-sdk/lib-storage`. It splits the stream into parts (5 MiB minimum, except the last) and uploads them concurrently. An S3 multipart upload allows at most 10,000 parts.

Key naming convention: `{type}/{userId}/{fileId}{ext}` — a server-generated `fileId` plus the validated extension, never the client-supplied filename (which the guidelines warn against: it enables path tricks and collisions). Keep presigned-URL lifetimes short — 300 s for uploads, 60–300 s for downloads — so a leaked URL expires quickly.

### 4. Upload Status Tracking

Database model:

```
files:
  id: UUID
  user_id: UUID
  original_name: string
  storage_key: string
  mime_type: string
  size_bytes: bigint
  status: enum(uploading, uploaded, processing, processed, failed)
  variants: jsonb (null until processed)
  error: text (null unless failed)
  created_at: timestamp
  updated_at: timestamp
```

### 5. API Endpoints

```
POST   /api/files/upload          — Multipart form upload (< 100MB)
POST   /api/files/presign         — Get presigned URL for large file upload
POST   /api/files/multipart/init  — Start multipart upload (> 100MB)
POST   /api/files/multipart/complete — Complete multipart upload
GET    /api/files/:id/status      — Get upload/processing status
GET    /api/files/:id/download    — Get presigned download URL
DELETE /api/files/:id             — Soft delete file
```

## Examples

### Example 1: Express Upload Endpoint

**Prompt**: "Create a file upload endpoint for my Express app. Accept images and PDFs, store in S3."

**Output**: Upload route with multer (memory storage, 10 MB `fileSize` limit), magic-byte validation on the buffer, S3 `PutObject`, database record creation, and error handling that returns 413 for oversized files. Returns file ID for status polling. Files above 10 MB go through the presigned path in Example 2.

### Example 2: Presigned Upload for Large Videos

**Prompt**: "Users upload videos up to 2GB. I don't want them going through my server."

**Output**: Presigned URL generation endpoint, client-side upload code with progress tracking, multipart upload for files > 100MB, and a webhook endpoint to confirm upload completion and trigger processing.

## Guidelines

- **Stream, don't buffer** — never load entire files into memory. multer's `memoryStorage()` buffers every file in RAM ("can cause your application to run out of memory"), so use `diskStorage()` or the `Upload` pipeline above for anything large
- **Validate magic bytes** — file extensions lie, magic bytes don't
- **Set upload limits at every layer** — nginx, reverse proxy, and application
- **Generate unique storage keys** — include user ID and file ID, never use original filename as key
- **Return immediately** — upload ack should be instant, processing happens async
- **Clean up on failure** — if DB write fails, delete the S3 object; if S3 fails, don't create DB record
- **Rate limit uploads** — per user, per time window (e.g., 20 uploads per hour)
Pliki w paczce (2)
  • SKILL.md 7 KB
  • _scores.json 1 KB

Wersje

W bazie nie ma innych skilli o nazwie file-upload-processor.

Migawka bazy z 13.11.2026 · wersja paczki 8ccd3f082d8d (skrót SHA-256 całej paczki)

Pobrania w ostatnich 30 dniach: 0 · ranking

Zgłoś problem z tym skillem: szkodliwa treść, błędny opis albo prośba autora o usunięcie.

Podobne skille

  • 100cloud-storage
    Integruje chmurowy object storage AWS S3, Azure Blob Storage i Google Cloud Storage: presigned URLs, multipart uploads, polityki bucketów, reguły cyklu życia i CDN.
  • 95file-handling
    Opisuje obsługę plików: przechowywanie w S3, R2 i Vercel Blob, generowanie PDF, import i eksport CSV i Excel, streaming, presigned URL i multipart upload.
  • 90upload-design
    Projektuje przesyłanie plików: pojedynczy lub wieloczęściowy upload, rozmiary części, podpisane adresy i bezpieczne nazwy obiektów.
  • 90role-fullstack:file-upload-handling
    Buduje wysyłanie plików z presigned URL, wznawianiem uploadu, walidacją MIME i skanowaniem wirusów, np. na S3 lub R2.
  • 100file-upload
    Obsługuje przesyłanie plików po stronie serwera w Multer, FastAPI UploadFile i Spring MultipartFile: walidacja, strumieniowanie, limity rozmiaru i strategie przechowywania.
  • 90file-storage-design
    Podpowiada przechowywanie plików użytkowników w object storage z presigned transferami, walidacją i regułami cyklu życia, przy budowie wgrywania i pobierania plików.

Uwagi i zgłoszenia

Wiadomość trafi do Bartosza Sokolińskiego. Adres e-mail jest potrzebny tylko do odpowiedzi. Szczegóły w polityce prywatności.