# BGCI STUDIO — Full Project Blueprint (Developer Perspectiv...

Prompt

# BGCI STUDIO — Full Project Blueprint (Developer Perspective) ### Platform Scope: Android (producer/source) + Windows (host/studio) only --- ## 1. Project Overview **BGCI Studio** turns an Android phone's camera into a professional webcam source for a Windows PC — positioned as a Camo/DroidCam competitor, but with one critical differentiator baked into the core architecture from day one: > **The Android device must connect and stream over USB *without* the user enabling "USB Debugging" (Developer Options / ADB).** This single constraint changes almost every downstream architecture decision (transport layer, permissions model, driver design), so it's treated as the central engineering problem of Phase 1 — not an afterthought. --- ## 2. The Core Engineering Challenge Most competitor apps (DroidCam, Iriun, EpocCam-style clones) use one of these approaches for USB connectivity: | Approach | Needs Developer Mode / ADB? | Why | |---|---|---| | ADB port forwarding (`adb forward`) | ✅ Yes | Requires USB debugging authorized | | MTP/PTP (file transfer mode) | ❌ No | But not designed for real-time video, high latency | | **USB Tethering (RNDIS/NCM)** | ❌ No | Creates a virtual network adapter over the USB cable — a normal *consumer* Android setting, not a developer setting | | **AOA — Android Open Accessory Protocol** | ❌ No | Native Android USB Accessory framework (Google-provided), designed exactly for third-party USB hosts talking to phones with zero developer flags | Since Phase 1 must avoid Developer Options entirely, BGCI Studio will standardize on **two non-ADB transport paths**, selectable automatically: 1. **Primary (recommended default): USB Tethering as a wired network transport** The user taps *Settings → Network → Hotspot & Tethering → USB Tethering* (a standard consumer toggle, not hidden in Developer Options). This creates a virtual Ethernet/RNDIS interface on both the phone and the PC. BGCI Studio then opens a normal TCP/UDP socket over that link — identical code path to Wi-Fi mode, just wired and lower-latency/jitter. 2. **Secondary (fallback/advanced): Android Open Accessory (AOA) protocol** The Windows app acts as a USB *host* using WinUSB/libusb, and the Android app runs a foreground service implementing Google's `UsbManager`/`UsbAccessory` API. AOA is a documented Android framework feature (not a debugging feature) — it lets any external device claim the phone as a USB accessory and exchange raw byte streams, no authorization dialog beyond a standard "Allow this app to access USB accessory?" prompt. Both are shipped in Phase 1 so users without USB tethering support (some carriers/OEMs restrict it) still get a wired option via AOA. > ⚠️ Wi-Fi Direct / LAN streaming is also included as the zero-cable option and reuses the *exact same protocol stack* as USB-tethering mode — only the transport socket differs. --- ## 3. High-Level System Architecture ``` ┌─────────────────────────────┐ Transport Layer ┌──────────────────────────────┐ │ ANDROID APP │ (choose one, same protocol) │ WINDOWS APP │ │ │ │ │ │ Camera2/CameraX Capture │ 1) Wi-Fi (TCP/UDP over LAN) │ Transport Receiver │ │ → YUV/NV21 Frame Buffer │ 2) USB Tethering (RNDIS/NCM) │ → Decoder │ │ → Hardware Encoder (H.264) │ 3) USB Accessory (AOA) │ → Frame Buffer │ │ → Packetizer │────────────────────────────────> → Virtual Camera Driver Sink │ │ → Transport Sender │ <── Control/Handshake channel ─│ → Studio UI (preview) │ │ Foreground Service (keeps │ │ → Windows OS exposes it as │ │ stream alive, battery-safe) │ │ a standard webcam device │ └─────────────────────────────┘ └──────────────────────────────────┘ ``` The Windows side is the "hub" — its output is a **virtual camera device** that any third-party app (Zoom, Teams, browsers, OBS) can select just like a normal webcam. --- ## 4. Android App — Technical Design **Core stack:** - **Language:** Kotlin (primary), minimal NDK/C++ for encoder pipeline performance - **Capture API:** CameraX (preferred for lifecycle-safety and lens abstraction) with fallback to Camera2 for advanced manual controls (exposure, focus, frame rate lock) needed later - **Encoding:** `MediaCodec` hardware-accelerated H.264 encoder (avoids CPU-bound software encoding, critical since we can't rely on desktop-grade GPUs) - **Transport abstraction layer:** a single `TransportInterface` with 3 implementations: - `WifiSocketTransport` (TCP for control, UDP or TCP for media) - `UsbTetherTransport` (binds socket to the RNDIS network interface specifically, detected via `ConnectivityManager.NetworkCallback`) - `AoaAccessoryTransport` (raw `FileInputStream`/`FileOutputStream` over `UsbAccessory` file descriptor) - **Foreground Service:** mandatory to keep camera + encoder alive when screen is off/app backgrounded, with a persistent notification (Android requirement for camera-using foreground services on Android 10+) - **Permission model:** Camera, Microphone (if audio in phase 1), Network, and `USB_PERMISSION` broadcast receiver for AOA — **no ADB, no root, no Developer Options required anywhere in the manifest or runtime flow** - **Discovery:** For Wi-Fi mode, use **NSD (Network Service Discovery / mDNS)** so the Windows app auto-finds the phone on the same LAN without manual IP entry - **Pairing/security:** On first connect, generate a short-lived pairing code or QR code shown on the PC screen and typed/scanned on the phone (prevents unauthorized devices from injecting video into a stranger's PC) --- ## 5. Windows App — Technical Design **Core stack:** - **Language:** C++ (performance-critical decode/driver path) + C#/.NET or WinUI 3 (UI layer), similar split to how Camo/OBS are built - **Decoding:** Media Foundation hardware decoder (`H264 Decoder MFT`) leveraging Intel Quick Sync / NVDEC / AMD VCN depending on GPU present - **Virtual Camera implementation — this is the single hardest Windows component.** Two realistic engineering paths: | Option | Description | Trade-off | |---|---|---| | **A. DirectShow Virtual Camera Filter** | Legacy but universally compatible (Zoom, older apps, OBS all support DirectShow sources) | More boilerplate, COM-heavy, needs code-signed driver install | | **B. Windows Media Foundation Frame Server (Frame Source)** | Modern API (Windows 10 2004+), what Microsoft recommends now, integrates with the new Camera app framework | Newer apps only; some legacy software still can't see MF-only virtual cams | **Recommendation for BGCI Studio:** ship **both** (a DirectShow filter for compatibility + an MF Frame Server for modern apps), same pattern Camo/OBS-VirtualCam use, unified behind one internal frame-delivery API so the capture/decode pipeline only needs to write frames to one internal buffer that both driver shims read from. - **USB Tethering handling on PC side:** Windows automatically creates an RNDIS/NCM network adapter when the phone tethers over USB — no custom Windows USB driver needed here at all; the app just needs to detect this new network interface (via `NetworkInformation` / `GetAdaptersAddresses`) and bind its socket listener to it preferentially over Wi-Fi if both are present. - **AOA host implementation:** requires **WinUSB** driver binding via a signed **INF file** (or bundling **libusb-win32/libusbK**) so the Windows app can claim the USB device in accessory mode and read/write bulk transfer endpoints directly. - **Installer requirements:** Windows driver components (DirectShow filter, MF extension, WinUSB binding) must be **digitally signed** (EV code-signing certificate) or Windows will block installation/loading — this is a real cost/logistics item to budget for early (Microsoft Hardware Dev Center attestation signing). --- ## 6. Communication Protocol Design (Transport-Agnostic Layer) A **custom lightweight binary protocol** is more efficient than reusing WebRTC/RTSP for this specific use case (lower overhead, no ICE/STUN needed since it's local link), structured as: - **Control Channel** (TCP, reliable): handshake, pairing/auth, resolution & frame-rate negotiation, keepalive/heartbeat, disconnect/reconnect events - **Media Channel** (TCP or UDP, chosen per transport type): H.264 NAL units, each frame chunked with a simple header — `[frame_id][timestamp][nal_type][length][payload]` - **Reconnection logic:** since phones sleep, apps get killed, and USB cables get bumped, the protocol needs automatic reconnect with state resume (don't force user to re-pair every drop) This same protocol runs unmodified over Wi-Fi sockets, USB-tethering sockets, or AOA byte streams — only the underlying transport class changes, which is the key architectural win of designing the transport abstraction this way. --- ## 7. Phase 1 Scope (MVP) — Explicit In/Out List Per your instruction, **Phase 1 excludes:** Streaming Features, Multi-Cam, Image Adjustments & Effects (these become Phase 2/3). **✅ Phase 1 IN-SCOPE:** 1. Android app: camera capture (single rear + front lens toggle), H.264 hardware encode, foreground service 2. Three working connection modes: Wi-Fi/LAN, USB Tethering, USB Accessory (AOA) — with automatic detection/fallback priority (USB > Wi-Fi for latency) 3. Device pairing & discovery (mDNS + pairing code) 4. Windows virtual camera driver (DirectShow + Media Foundation) delivering raw decoded frames to any app that lists webcams 5. Minimal Windows desktop UI: device list, connect/disconnect, live preview, resolution/FPS selector, basic mirror/rotate toggle 6. Reconnection & error-handling (cable unplug, app kill, network drop) 7. Basic audio passthrough (phone mic → PC virtual microphone) — optional stretch, since many meeting use-cases need mic too 8. Logging/diagnostics module for support (no telemetry beyond opt-in crash logs) **❌ Explicitly OUT of Phase 1** (per your list): live streaming to YouTube/Twitch/RTMP, multi-camera scene composition, filters/LUTs/background replacement/portrait bokeh, overlays. --- ## 8. Technology Stack Summary | Layer | Android | Windows | |---|---|---| | Language | Kotlin + NDK(C++) | C++ / C# (.NET) | | Capture | CameraX / Camera2 | — | | Encode/Decode | MediaCodec (HW H.264) | Media Foundation MFT (HW decode) | | Transport | Sockets (TCP/UDP), UsbAccessory API, ConnectivityManager | Winsock, WinUSB/libusb, Network adapter detection | | Virtual output | — | DirectShow filter + MF Frame Server, signed driver | | UI | Jetpack Compose | WinUI 3 / WPF | | Discovery | NSD (mDNS) | Bonjour/mDNS client (e.g., Zeroconf lib) | | Build/CI | Gradle, Android CI | MSBuild, Windows CI, EV code-signing pipeline | --- ## 9. Team Composition Needed - 1× Android engineer (Camera2/CameraX + MediaCodec + USB Accessory experience — this is a niche skill, worth screening for specifically) - 1× Windows systems engineer (DirectShow/Media Foundation driver development — also niche; prior virtual-camera or OBS-plugin experience is gold) - 1× Networking engineer (socket protocol design, NAT/adapter binding, reconnection logic) — can overlap with either above role - 1× UI/UX developer (WinUI/Compose) - 1× QA engineer (device-matrix testing — Android fragmentation across OEM USB tethering implementations is a real risk) - 1× Project lead/architect to own the protocol spec and driver-signing logistics --- ## 10. Key Risks & Mitigations | Risk | Mitigation | |---|---| | OEM Android skins block/hide USB Tethering toggle | Detect and show in-app guided instructions per-OEM; fallback to AOA | | Some Android devices don't support AOA (older/budget chipsets) | Wi-Fi mode always available as universal fallback | | Windows driver signing rejected/unsigned driver blocked by SmartScreen/Secure Boot | Budget for EV code-signing cert + Microsoft Hardware Dev Center attestation early, not late | | Latency/jitter on Wi-Fi in congested networks | Prioritize USB paths automatically when detected; add adaptive bitrate later | | USB tethering interface conflicts with existing PC network config | Explicit adapter selection UI + auto-detect RNDIS interface by vendor ID | --- ## 11. Suggested Phase 1 Timeline (rough, engineering-only estimate) | Milestone | Duration | |---|---| | Protocol spec + architecture finalization | 2–3 weeks | | Android capture/encode pipeline (Wi-Fi only first) | 3–4 weeks | | Windows decode + basic virtual cam (DirectShow only first) | 4–5 weeks | | USB Tethering transport integration (both sides) | 2–3 weeks | | AOA transport integration (both sides) | 3–4 weeks (higher complexity) | | Media Foundation Frame Server (modern virtual cam) | 2 weeks | | Pairing/discovery/reconnect hardening | 2 weeks | | Device-matrix QA + driver signing/certification | 3 weeks | | **Total Phase 1** | **~4–5 months** with the team above | --- Would you like me to go deeper into any single piece next — e.g., the exact **AOA handshake sequence**, the **DirectShow virtual camera filter code structure**, or the **binary protocol packet spec**?