Skip to content
Skills Library
Programming · Building & coding

prototypekit

/prototypekit

Reference and code-generation guide for PrototypeKit, a SwiftUI framework for rapid on-device machine-learning prototyping on iOS/macOS. Use whenever the user is building with PrototypeKit or asks for a live camera feed, on-device image classification, object detection, live text/OCR recognition, barcode scanning, animal recognition, face detection, body pose detection, rectangle detection, hand-pose classification, sound recognition, or natural-language analysis (sentiment, language identification, named-entity recognition) in a SwiftUI app. Provides exact initializer signatures, @State/@Bind

Licence
Apache-2.0
Last repository change
02.07.2026

Assessment and security

95/ 100Class: good
Format25 / 25
Frontmatter with name and description, valid name, description up to 1,024 characters, name matching the folder.
Discoverability25 / 25
Description explains when to use the skill, has a reasonable length and uses third person.
Content30 / 35
At least 150 words and at most 500 lines, headings, step lists, examples or code.
Hygiene15 / 15
No placeholders such as TODO, no Windows paths, with a licence.
Riskno signals detected in the scan
Files in this package
1
Scripts
0
HTML comments
0
URLs in SKILL.md
2
Deprecation

No signals detected. Repository changed on 02.07.2026, without references to retired models or APIs.

Licence

Apache-2.0, otwarta: można używać i zmieniać, z zachowaniem licencji i informacji o zmianach.

What it does and when to use it

Reference and code-generation guide for PrototypeKit, a SwiftUI framework for rapid on-device machine-learning prototyping on iOS/macOS. Use whenever the user is building with PrototypeKit or asks for a live camera feed, on-device image classification, object detection, live text/OCR recognition, barcode scanning, animal recognition, face detection, body pose detection, rectangle detection, hand-pose classification, sound recognition, or natural-language analysis (sentiment, language identification, named-entity recognition) in a SwiftUI app. Provides exact initializer signatures, @State/@Bind

Original author description

Reference and code-generation guide for PrototypeKit, a SwiftUI framework for rapid on-device machine-learning prototyping on iOS/macOS. Use whenever the user is building with PrototypeKit or asks for a live camera feed, on-device image classification, object detection, live text/OCR recognition, barcode scanning, animal recognition, face detection, body pose detection, rectangle detection, hand-pose classification, sound recognition, or natural-language analysis (sentiment, language identification, named-entity recognition) in a SwiftUI app. Provides exact initializer signatures, @State/@Bind

How it works

  1. Installation (Swift Package Manager)
  2. Required Info.plist permissions (most common cause of crashes)
  3. Core ML models
  4. API reference
  5. Gotchas
Original SKILL.md headings · 572 lines, 1,765 words
SKILL.md preview
---
name: prototypekit
description: >-
  Reference and code-generation guide for PrototypeKit, a SwiftUI framework for
  rapid on-device machine-learning prototyping on iOS/macOS. Use whenever the
  user is building with PrototypeKit or asks for a live camera feed, on-device
  image classification, object detection, live text/OCR recognition, barcode
  scanning, animal recognition, face detection, body pose detection, rectangle
  detection, hand-pose
  classification, sound recognition, or natural-language analysis (sentiment,
  language identification, named-entity recognition) in a SwiftUI app. Provides exact
  initializer signatures, @State/@Binding wiring, required Info.plist privacy
  keys, and Core ML model setup.
---

# PrototypeKit

PrototypeKit wraps Vision, AVFoundation, SoundAnalysis, and Core ML behind a
handful of drop-in SwiftUI views and one view modifier, so you can prototype
on-device ML apps quickly. It is intentionally a simplified interface — great for
idea validation, not for production.

- **Platforms:** iOS 14+ and macOS 13+ (some features are iOS-only — see below).
- **Import:** `import PrototypeKit`
- **Distribution:** Swift Package Manager.

Always use the exact signatures in this document — do not invent parameters.

## Installation (Swift Package Manager)

In Xcode: **File ▸ Add Package Dependencies…** and enter
`https://github.com/FridayTechnologies/PrototypeKit`.

Or in a `Package.swift`:

```swift
dependencies: [
    .package(url: "https://github.com/FridayTechnologies/PrototypeKit", branch: "master")
],
targets: [
    .target(name: "MyApp", dependencies: ["PrototypeKit"])
]
```

Then `import PrototypeKit` in any file that uses it.

## Required Info.plist permissions (most common cause of crashes)

Any camera-based view (`PKCameraView`, `ImageClassifierView`, `ObjectDetectorView`,
`LiveTextRecognizerView`, `LiveBarcodeRecognizerView`, `LiveAnimalRecognizerView`,
`LiveFaceDetectorView`, `LiveBodyPoseDetectorView`, `LiveRectangleDetectorView`,
`HandPoseClassifierView`, `ActionClassifierView`) requires a camera-usage description, or the app
crashes on launch of that view:

- **Camera:** key `NSCameraUsageDescription` — shown in Xcode's Info tab as
  `Privacy - Camera Usage Description`. Value: a human-readable reason.

The sound modifier requires microphone access:

- **Microphone:** key `NSMicrophoneUsageDescription` —
  `Privacy - Microphone Usage Description`.

To add: select the project ▸ your target ▸ **Info** tab ▸ right-click the
"Custom iOS Target Properties" table ▸ **Add Row** ▸ pick the key ▸ type the reason.

## Core ML models

Views that classify or detect (`ImageClassifierView`, `ObjectDetectorView`,
`HandPoseClassifierView`, `ActionClassifierView`) take a `modelURL: URL` pointing at a compiled
Core ML model.

1. Drag your `.mlmodel` (from Create ML / Core ML) into the Xcode project.
2. Xcode generates a Swift class named after the model.
3. Pass its bundled URL, e.g. `FruitClassifier.urlOfModelInThisBundle`.

## API reference

### `PKCameraView` — raw live camera feed

```swift
PKCameraView()
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct ContentView: View {
    var body: some View {
        VStack {
            PKCameraView()
        }
        .padding()
    }
}
```

### `ImageClassifierView` — live image classification (Core ML)

Signature:

```swift
public init(modelURL: URL,
            latestPrediction: Binding<String> = .constant(""),
            camera: CameraOptions? = nil)
```

`CameraOptions` is a typealias:

```swift
public typealias CameraOptions = (position: AVCaptureDevice.Position,
                                  deviceType: AVCaptureDevice.DeviceType)
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct ImageClassifierViewSample: View {
    @State var latestPrediction: String = ""

    var body: some View {
        VStack {
            ImageClassifierView(modelURL: FruitClassifier.urlOfModelInThisBundle,
                                latestPrediction: $latestPrediction)
            Text(latestPrediction)
        }
    }
}
```

### `ObjectDetectorView` — live object detection (Core ML)

Runs a Create ML / Core ML **Object Detector** model on each camera frame and publishes the
objects found. There are two initializers — pick by whether you need only *what* was detected or
also *where*:

```swift
// Labels only:
public init(modelURL: URL,
            detectedObjects: Binding<[String]> = .constant([]),
            camera: CameraOptions? = nil,
            onError: ((PrototypeKitError) -> Void)? = nil)

// Labels + confidence + bounding boxes:
public init(modelURL: URL,
            detectedObjects: Binding<[DetectedObject]>,
            camera: CameraOptions? = nil,
            onError: ((PrototypeKitError) -> Void)? = nil)
```

`DetectedObject` is a public value type:

```swift
public struct DetectedObject: Equatable {
    public let label: String       // top Vision label
    public let confidence: Float   // 0...1
    public let boundingBox: CGRect // normalized (0...1), origin bottom-left (Vision convention)
}
```

The overload is chosen by the binding's element type: pass `Binding<[String]>` for labels only, or
`Binding<[DetectedObject]>` for positions. Vision's bounding-box origin is bottom-left, so flip the
`y` axis (`1 - boundingBox.midY`) when positioning a SwiftUI overlay.

Full example (labels only):

```swift
import SwiftUI
import PrototypeKit

struct ObjectDetectorViewSample: View {
    @State var detectedObjects: [String] = []

    var body: some View {
        VStack {
            ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
                               detectedObjects: $detectedObjects)

            ScrollView {
                ForEach(Array(detectedObjects.enumerated()), id: \.offset) { index, object in
                    Text(object)
                }
            }
        }
    }
}
```

Full example (bounding boxes):

```swift
import SwiftUI
import PrototypeKit

struct ObjectDetectorBoxesSample: View {
    @State var detectedObjects: [DetectedObject] = []

    var body: some View {
        ZStack {
            ObjectDetectorView(modelURL: MyObjectDetector.urlOfModelInThisBundle,
                               detectedObjects: $detectedObjects)

            GeometryReader { geometry in
                ForEach(Array(detectedObjects.enumerated()), id: \.offset) { _, object in
                    let box = object.boundingBox
                    Rectangle()
                        .stroke(.red, lineWidth: 2)
                        .frame(width: box.width * geometry.size.width,
                               height: box.height * geometry.size.height)
                        .position(x: box.midX * geometry.size.width,
                                  y: (1 - box.midY) * geometry.size.height)
                        .overlay(Text(object.label))
                }
            }
        }
    }
}
```

### `LiveTextRecognizerView` — live text recognition / OCR

Signature:

```swift
public init(detectedText: Binding<[String]> = .constant([]))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct TextRecognizerView: View {
    @State var detectedText: [String] = []

    var body: some View {
        VStack {
            LiveTextRecognizerView(detectedText: $detectedText)

            ScrollView {
                ForEach(Array(detectedText.enumerated()), id: \.offset) { line, text in
                    Text(text)
                }
            }
        }
    }
}
```

### `LiveBarcodeRecognizerView` — live barcode / QR scanning

Signature:

```swift
public init(detectedBarcodes: Binding<[String]> = .constant([]))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct BarcodeRecognizerView: View {
    @State var detectedBarcodes: [String] = []

    var body: some View {
        VStack {
            LiveBarcodeRecognizerView(detectedBarcodes: $detectedBarcodes)

            ScrollView {
                ForEach(Array(detectedBarcodes.enumerated()), id: \.offset) { index, barcode in
                    Text(barcode)
                }
            }
        }
    }
}
```

### `LiveAnimalRecognizerView` — live animal recognition (cats & dogs)

Uses the built-in Vision animal recognizer — no Core ML model required.

Signature:

```swift
public init(detectedAnimals: Binding<[String]> = .constant([]))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct AnimalRecognizerView: View {
    @State var detectedAnimals: [String] = []

    var body: some View {
        VStack {
            LiveAnimalRecognizerView(detectedAnimals: $detectedAnimals)

            ScrollView {
                ForEach(Array(detectedAnimals.enumerated()), id: \.offset) { index, animal in
                    Text(animal)
                }
            }
        }
    }
}
```

### `LiveFaceDetectorView` — live face detection

Uses the built-in Vision face detector — no Core ML model required. Publishes the number
of faces found in the latest frame.

Signature:

```swift
public init(faceCount: Binding<Int> = .constant(0))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct FaceDetectorView: View {
    @State var faceCount: Int = 0

    var body: some View {
        VStack {
            LiveFaceDetectorView(faceCount: $faceCount)
            Text("Faces: \(faceCount)")
        }
    }
}
```

### `LiveBodyPoseDetectorView` — live human body pose detection

Uses the built-in Vision human body pose request — no Core ML model required. Publishes the
number of bodies found in the latest frame.

Signature:

```swift
public init(bodyCount: Binding<Int> = .constant(0))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct BodyPoseDetectorView: View {
    @State var bodyCount: Int = 0

    var body: some View {
        VStack {
            LiveBodyPoseDetectorView(bodyCount: $bodyCount)
            Text("Bodies: \(bodyCount)")
        }
    }
}
```

### `LiveRectangleDetectorView` — live rectangle detection

Uses the built-in Vision rectangle detector — no Core ML model required. Detects rectangular
shapes (documents, cards, signs) and publishes the count found in the latest frame.

Signature:

```swift
public init(rectangleCount: Binding<Int> = .constant(0))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct RectangleDetectorView: View {
    @State var rectangleCount: Int = 0

    var body: some View {
        VStack {
            LiveRectangleDetectorView(rectangleCount: $rectangleCount)
            Text("Rectangles: \(rectangleCount)")
        }
    }
}
```

### `HandPoseClassifierView` — hand-pose classification (Core ML)

Detects a hand pose with Vision and classifies keypoints with your Core ML model.

Signature:

```swift
public init(modelURL: URL, latestPrediction: Binding<String> = .constant(""))
```

Full example:

```swift
import SwiftUI
import PrototypeKit

struct HandPoseSample: View {
    @State var latestPrediction: String = ""

    var body: some View {
        VStack {
            HandPoseClassifierView(modelURL: HandPoseClassifier.urlOfModelInThisBundle,
                                   latestPrediction: $latestPrediction)
            Text(latestPrediction)
        }
    }
}
```

### `ActionClassifierView` — action classification from body movement (Core ML)

Detects a person's body-pose keypoints with Vision and classifies their action from a sliding
window of frames using a Create ML / Core ML **Action Classifier** model. An action unfolds over
time, so the view collects a window (two seconds by default) before its first prediction and
updates as the window advances.

Signature:

```swift
public init(modelURL: URL,
            configuration: ActionClassifierConfiguration = .init(),
            latestPrediction: Binding<String> = .constant(""),
            camera: CameraOptions? = nil,
            onError: ((PrototypeKitError) -> Void)? = nil)
```

`ActionClassifierConfiguration` exposes `posesFeatureName` (default `"poses"`), `labelFeatureName`
(default `"label"`), `predictionWindowSize` (default `60`), and `predictionInterval` (default `15`)
for models that differ from the Create ML template.

Full example:

```swift
import SwiftUI
import PrototypeKit

struct ActionClassifierSample: View {
    @State var latestPrediction: String = ""

    var body: some View {
        VStack {
            ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle,
                                 latestPrediction: $latestPrediction)
            Text(latestPrediction)
        }
    }
}
```

### `.recognizeSounds` — live sound recognition (iOS 15+ only)

A `View` modifier. Uses the built-in system sound classifier by default, or a
custom Core ML model via the configuration.

Signature:

```swift
@available(iOS 15.0, *)
public func recognizeSounds(recognizedSound: Binding<String?>,
                            configuration: SoundAnalysisConfiguration = .init()) -> some View
```

Configuration:

```swift
public struct SoundAnalysisConfiguration {
    public init(inferenceWindowSize: Double = 1.5,  // seconds of audio per prediction
                overlapFactor: Double = 0.9,        // 0–1 overlap between windows
                mlModel: MLModel? = nil)            // nil = system classifier
}
```

Full example (system classifier):

```swift
import SwiftUI
import PrototypeKit

struct SoundRecognizerView: View {
    @State var recognizedSound: String?

    var body: some View {
        VStack {
            Text("Recognized Sound: \(recognizedSound ?? "None")")
        }
        .recognizeSounds(recognizedSound: $recognizedSound)
    }
}
```

Custom model + configuration:

```swift
.recognizeSounds(
    recognizedSound: $recognizedSound,
    configuration: SoundAnalysisConfiguration(
        inferenceWindowSize: 1.5,
        overlapFactor: 0.9,
        mlModel: yourCustomModel   // an MLModel instance
    )
)
```

### Natural Language — text analysis (no camera, mic, permissions, or model)

Three `View` modifiers wrap Apple's `NaturalLanguage` framework. They need no camera,
microphone, privacy keys, or Core ML model, and work on **both iOS and macOS**. Each
re-runs whenever the `text` you pass changes, updating a binding with the result.

Signatures:

```swift
public func analyzeSentiment(text: String, score: Binding<Double>) -> some View
public func identifyLanguage(text: String, language: Binding<String?>) -> some View
public func tagEntities(text: String, entities: Binding<[String]>) -> some View
```

- `analyzeSentiment` — `score` ranges from `-1` (very negative) to `1` (very positive); `0` is neutral.
- `identifyLanguage` — `language` is a BCP-47 code (e.g. `"en"`, `"fr"`), or `nil` if undetermined.
- `tagEntities` — `entities` are the recognized people, place, and organization names.

Full example (sentiment):

```swift
import SwiftUI
import PrototypeKit

struct SentimentView: View {
    @State var text: String = "I love this!"
    @State var score: Double = 0

    var body: some View {
        VStack {
            TextField("Type something", text: $text)
            Text("Sentiment: \(score, specifier: "%.2f")")
        }
        .analyzeSentiment(text: text, score: $score)
    }
}
```

## Gotchas

- **Missing privacy key = crash.** Add `NSCameraUsageDescription` (camera views)
  and `NSMicrophoneUsageDescription` (`recognizeSounds`) before running.
- **Bad model URL.** `ImageClassifierView` / `ObjectDetectorView` /
  `HandPoseClassifierView` / `ActionClassifierView` degrade gracefully if the model fails to load — the camera
  feed still shows but no predictions/detections are produced, and the failure is logged
  (and reported to `onError` where available). Verify the model is in the target's bundle
  and the generated class name matches.
- **Simulator limits.** The camera isn't available in the iOS Simulator (limited
  preview), and inference falls back to CPU-only in the simulator and under XCTest.
- **`recognizeSounds` is iOS-only** and requires iOS 15+. It is not available on
  macOS. Gate calls with `if #available(iOS 15.0, *)` where needed.
- Prediction/detection values arrive asynchronously through the `@Binding`s; drive
  other SwiftUI views off those state variables as usual.
- **Natural Language modifiers** (`analyzeSentiment`, `identifyLanguage`, `tagEntities`)
  are the exception to most of the above: no camera/mic, no privacy keys, no Core ML
  model, and they run on macOS too. They re-analyse whenever the `text` argument changes.

Versions

The database has no other skills named prototypekit.

Database snapshot: 13.11.2026 · package version 20378096c0d0 (SHA-256 digest of the entire package)

Downloads in the last 30 days: 0 · ranking

Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.

Similar skills

  • 95vision-framework
    Implement computer vision features including text recognition (OCR), face detection, barcode scanning, image segmentation, object tracking, and document scanning in iOS apps. Covers both the modern Swift-native Vision API (iOS 16+) and legacy VNRequest patterns, VisionKit…
  • 95natural-language
    Tokenize, tag, and analyze natural language text using Apple's NaturalLanguage framework and translate between languages with the Translation framework. Use when adding language identification, sentiment analysis, named entity recognition, part-of-speech tagging, text…
  • 100core-ml
    Core ML, Create ML, Vision framework, Natural Language framework, on-device ML integration. Use when user wants image classification, text analysis, object detection, sound classification, model optimization, or custom model integration. Covers Core ML vs Foundation Models decision.
  • 100swiftui-review
    Reviews SwiftUI code for best practices on modern APIs, maintainability, and performance. This skill should be used when the user asks to review SwiftUI code, check for deprecated iOS/macOS APIs, validate data flow patterns, or audit accessibility compliance in Swift projects.
  • 100swiftui-expert-skill
    Write, review, or improve SwiftUI code following best practices for state management, view composition, performance, macOS-specific APIs, and iOS 26+ Liquid Glass adoption. Use when building new SwiftUI features, refactoring existing views, reviewing code quality, or adopting…
  • 100swiftui-patterns
    SwiftUI architecture patterns, state management with @Observable, view composition, navigation, performance optimization, and modern iOS/macOS UI best practices. Use when building or reviewing SwiftUI views, @Observable state, navigation, or render performance.

Feedback and reports

Your message goes to Bartosz Sokoliński. An email address is needed only for a reply. See the privacy policy.