Technology
On-device AI at Movelytics
On-device AI means the model runs where the data is captured: on the user's phone, or on the customer's own infrastructure. At Movelytics the camera frame, the keypoints, the joint angles and the rep count are all computed on the phone, in real time. The only thing that reaches us is a usage count for billing.
Last updated:
Vertical jump · knee and ankle anglesIllustration. Authored pose sequence, not live inference.
01Definition
What on-device inference means
Inference is the moment a trained model is used: an input goes in, a result comes out. On-device inference means that this computation happens on the hardware that captured the input. For a movement app, the phone that films the user is also the computer that analyses the film.
The alternative is cloud inference. The app records a video, uploads it, a server runs the model, and the result comes back. That design is simple to build and it works for analysis after the fact. It does not work when a person needs feedback during the movement, when the connection is poor, or when the video should not leave the phone at all.
We chose the first design for every product that looks at a body. It gives three properties that are hard to get any other way:
- Real time. A result arrives for each frame, with no network round trip. A rep is counted when it ends, not a few seconds later.
- Privacy by construction. The video is never sent to us, so there is no video for us to store, leak or sell.
- A cost that does not grow with video minutes. The computation runs on hardware the user already owns.
The price is engineering. A phone has a small processor, a battery and a thermal limit, and there are thousands of phone models. Most of our work goes there.
02Pipeline
From camera to rep count, without leaving the phone
A movement feature is a chain of four steps. Each one takes the output of the previous one, and all four run on the device.
Everything in this frame runs on the phone
01 Camera
Frame
The camera delivers an image. It stays in the phone's memory.
02 Model
Keypoints
The pose model turns the image into named points on the body.
03 Geometry
Angles
Joint angles are computed from the keypoints, frame after frame.
04 Logic
Reps and scores
Rules on angles over time give repetitions, holds and a form score.
Stays on the deviceVideo frames, keypoints, angles, reps and scores. The host app decides what to keep.
Reaches MovelyticsA usage count: the number of API calls, used for billing. Nothing from the camera.
- Camera. The phone delivers frames. They are resized and cropped around the person to the model's input size (192 pixels for the PoseTracker Model).
- Model. A pose estimation model reads the frame and returns keypoints: named points on the body, each with a position and a confidence score.
- Geometry. From three keypoints you get a joint angle. From the angle over time you get a range of motion, a speed, a symmetry.
- Logic. Rules on those signals produce what the app shows: a repetition counted, a hold timed, a form score, a jump height, a flexibility measure.
The host app receives structured events and decides what to do with them: update the screen, store a session, trigger the next exercise. PoseTracker does not keep an analysis history for end users. The app owns it.
03Low-end Android
Why low-end Android is the hard part
A recent flagship phone runs almost any small model in real time. That tells you little about your users. Many people keep a phone for five years, and many of those phones were entry-level when they were bought.
On these devices three things go wrong at once. The processor has slow cores and little or no usable acceleration for neural networks. The camera pipeline itself takes a large share of each frame's time budget. And heat builds up: after a few minutes the system slows the processor down, so a model that held 30 frames per second in a short test drops below 20 in a real session.
This is why we measure on a low-end phone first, for ten minutes, with the camera on. In our benchmark of 17 November 2026, on a Huawei FIG-LX1 2017 running Android 9, the PoseTracker Model takes 18.4 ms per frame alone and holds 28.7 frames per second with the camera over 10 minutes (slowest 10-second window: 22.1). These figures are preliminary and come from an internal build, not from a shipped model. The method, the comparison with public models and the limits are on the benchmarks page.
04Runtime
Native runtime versus WebView
A model can run on a phone in two ways. In a WebView, the model runs inside an embedded browser, with web technologies. In a native runtime, it runs as compiled code that talks to the processor directly.
The WebView route has real advantages: one implementation for iOS, Android and the web, a single line to integrate, and nothing heavy to add to the app. It is how a team gets movement tracking into an app in minutes. Its cost is overhead. The browser layer takes time on every frame, and on a low-end phone that time is the difference between smooth and choppy.
The native route removes that overhead and gives control over threads and memory. It is where the PoseTracker Model is designed to run. We keep both paths: the fast integration where it is good enough, the native runtime where every millisecond counts.
| WebView | Native runtime | |
|---|---|---|
| Integration effort | Very low: one embedded view | Higher: a native module per platform |
| Per-frame overhead | Browser layer on every frame | Close to the model's own time |
| Low-end Android | Often the limiting factor | Where real time becomes reachable |
| Best for | Fast start, web apps, recent phones | Large fleets, older phones, long sessions |
058-bit inference
8-bit inference, in plain words
A neural network is a long list of numbers. They are usually stored as 32-bit floating-point values. 8-bit inference, also called quantised inference, stores and computes them as 8-bit integers instead. The model file is about four times smaller, and integer arithmetic is what entry-level processors do fastest.
The catch is precision. Rounding every number to one of 256 levels can degrade the result, sometimes badly, unless the model is trained and exported with that constraint in mind. A model designed for 8-bit from the start keeps its accuracy; a model converted as an afterthought often does not.
The effect on speed is visible on public models too. On the Huawei FIG-LX1 2017, MoveNet Lightning takes 38.6 ms per frame in floating point and 26.2 ms in 8-bit, model alone. Our research note on 8-bit inference on entry-level phones goes through the numbers.
06Data
What we meter, and what we never see
Running on the device changes what a vendor can know. We designed our products so that the answer is: very little.
- What we meter
- The number of API calls made by an app. It is the basis of billing. It contains no image and no movement data.
- What we never receive
- The camera video, individual frames, keypoints, joint angles, rep counts, scores, or any history of a person's sessions.
- Who keeps the data
- The app that uses our tools. If it stores sessions, it does so in its own system, under its own privacy policy.
- What we never do
- Sell data. There is no movement dataset built from our customers' users.
The full list of commitments, in plain language, is on the data commitments page.
07Beyond pose
On-device beyond pose: language models and hardware
The same discipline applies to other kinds of models. Two directions matter to the companies we work with.
Domain language models, on device or on premises
A physiotherapy practice or a rehabilitation programme may want a language model that knows its protocols and vocabulary, and that never sends a patient note to a third party. We build domain-specific language models sized to run on a device or on the customer's own servers. "On premises" means inside infrastructure the customer controls.
Hardware plus software
Some products are objects: a mirror, a training machine, a kiosk. The model then runs on an embedded computer with its own camera, its own thermal limits and no app store. We take on these projects as a lab. They are covered by non-disclosure agreements, so we do not name or describe them.
If you want the developer's view of running pose estimation on the device, with code and package details, the PoseTracker guide on on-device pose estimation covers it. To choose between public pose models, see PoseTracker's comparison of MediaPipe and MoveNet.
08FAQ
Questions about the technology
Does on-device AI need an internet connection?
The model does not need a network to run: the frame is analysed on the phone. A connection is used for what surrounds the model, such as loading the app and reporting the usage count used for billing.
Which phones can run Movelytics models in real time?
We optimise for the phones people actually own, including low-end and five-year-old Android devices, and for iOS. In our preliminary benchmark of 17 November 2026, an internal build of the PoseTracker Model held 28.7 frames per second with the camera on a Huawei FIG-LX1 2017 (Android 9) over 10 minutes (slowest 10-second window: 22.1).
What is 8-bit inference?
It is a way to run a neural network with 8-bit integers instead of 32-bit floating-point numbers. The model is about four times smaller and runs faster on entry-level processors. It keeps its accuracy only if the model was designed for it.
Can Movelytics run a language model on our own servers?
Yes. The lab builds domain-specific language models that run on a device or on premises, inside infrastructure the customer controls. See work with the lab.
09Go further
Next step
See what runs on this stack.
Three products use it today. The lab can also build on it for you.