feat: Add camera livestream and voice call support#764
feat: Add camera livestream and voice call support#764clawd139 wants to merge 1 commit intoPython-roborock:mainfrom
Conversation
Adds a standalone camera module for streaming video/audio from Roborock vacuums with cameras. This addresses part of Python-roborock#738. Features: - Live video streaming (HD/SD quality) - Bidirectional voice calls (speak to and hear from robot) - Snapshot capture - Video recording - Audio sending for intercom functionality The implementation uses: - MQTT for WebRTC signaling (via existing protocol encoders) - aiortc for WebRTC peer connection - paho-mqtt for MQTT connection (standalone, not using aiomqtt) This is provided as a standalone module that could be integrated more tightly with the library's session management in future work. See docs/CAMERA_PROTOCOL.md for full protocol documentation. New dependencies (optional): - aiortc: WebRTC implementation - paho-mqtt: MQTT client - pyaudio: Voice call audio (optional) - opencv-python: Video display (optional)
|
Impressive result and use of AI. I know you said this is a standalone module, but not sure we should necessarily add a standalone module that doesn't really fit into the code base structure? Happy to discuss with you on discord if you do intend to make this fit in with the project. |
Agreed! We should utilize the existing mqtt topic rather than making a new one. Just so I can understand, what did your base this off of? Did you grab the mitm packets yourself? Did you use https://github.com/AlexxIT/go2rtc/tree/master/pkg/roborock or did you use the decompiled app code? I do think we may be able to do these mqtt commands over the local connection as well (not sure for these, but worth trying) Then maybe to make it easier to review - once the logic is more integrated with our existing logic, let's remove the ability to talk to the stream for now to minimize functionality being added and make it a bit easier to review, then that can be added in a follow up! Thanks for tackling this! It has been on my todo list for a long time! |
Hey! I'm the bot owner. clawd139 is my OpenClaw instance. This was a couple hrs of back-and-forth but it did 99% of the work. I just helped with grabbing mitm packets. I know this wasn't built with python-roborock compatibility in mind but hoping the protocol documentation helps :) Edit: the decompiled apk was also used to help debug |
Summary
This PR adds a standalone camera module for streaming video/audio from Roborock vacuums with cameras. Addresses part of #738 (Camera Livestream, Remote Control).
Features
Implementation
The module uses:
roborock.protocolencodersThis is provided as a standalone module (
roborock/camera.py) that manages its own MQTT connection. Future work could integrate it more tightly with the library's session management and aiomqtt usage.Protocol Documentation
Full protocol documentation is included in
docs/CAMERA_PROTOCOL.md, covering:Dependencies
New optional dependencies:
aiortc- WebRTC implementationpaho-mqtt- MQTT clientpyaudio- Voice call audio playback (optional)opencv-python- Video display (optional)Usage Example
Tested On
Other camera-equipped models likely work but are untested.
Notes
start_voice_chatcommand must be called to receive audio framesHappy to iterate on this if you'd prefer different structuring or tighter integration with the existing codebase!
Closes part of #738