---
id: f06329e9-7e81-41f8-acf2-a48baf70cdcd
title: 'build aborts (allocator) on large blobs: every file is read fully into memory'
state: closed
labels:
- robustness
assignees: []
milestone: null
created: 2026-10-27T14:54:10.641778270Z
updated: 2026-10-27T15:41:26.855301643Z
---
`build` reads every blob fully into memory (`gitx::show` → `run_git` → `Vec<u8>`), regardless of size. `PREVIEW_MAX_BYTES` (256 KiB) only decides whether the *page* renders a preview — the bytes were already in RAM.

Verified:

```
$ head -c 400000000 /dev/zero > big.bin && git add -A && git commit -m big
$ ( ulimit -v 900000; hu13 build . -o out )
memory allocation of 400000000 bytes failed
Aborted (core dumped)          # exit 134
```

The failure is a raw allocator abort (SIGABRT), not a clean `hu13: ...` error, and in `entrypoint.sh` this means the container crash-loops before serve starts.

Real repos carry large binaries (videos, datasets, vendored archives) and a PR can add one deliberately. There is no `--max-blob` and no streaming path.

Fix direction:
- `git cat-file --batch` with a size probe (`git cat-file -s <rev>:<path>` first) so blobs above a threshold are never read; write a placeholder page and skip/stream the `raw/` copy
- or stream `git show` straight into the `raw/` file and only keep the first 256 KiB for preview/binary sniff
- make the threshold configurable (`--max-blob`, `HU13_MAX_BLOB`) with a documented default (say 10 MiB) and a clear warning line when a file is skipped
- ensure the failure mode is a clean error, not an allocator abort

Acceptance:
- [ ] a repo with a 500 MB blob builds (skips it, warns) instead of aborting
- [ ] peak RSS stays bounded by the threshold, not by the largest blob
- [ ] `raw/` either contains the full file (streamed) or is documented as skipped