Skip to content

Put large binary files under Git LFS without wrecking your repository

Learn how to keep large binary files such as videos and design assets in Git LFS without bloating your repository. This guide explains how the clean and smudge filters swap each file for a small pointer file, how to set tracking patterns in .gitattributes, and how to move files already in history with git lfs migrate import. It also covers CI clones with GIT_LFS_SKIP_SMUDGE, file locking and storage quotas.

This guide shows developers how to keep large binary files, such as videos, images and design assets, in a Git repository with Git LFS (Large File Storage). Git stores every version of every file whole, forever. Commit a 240 MB video six times and the pack files (Git’s compressed storage) carry roughly 1.4 GB that every clone downloads, on every machine. Git LFS fixes this by intercepting the file on its way into the index, Git’s staging area. The commit gets a text pointer of about 130 bytes, and the real bytes go to a separate LFS server. A checkout pulls them back only when it needs them. Two things trip nearly everyone. The .gitattributes file that defines which patterns are tracked is itself a normal tracked file and must be committed, or nobody else’s clone applies the filters. And enabling LFS does nothing to what is already in history — old blobs (stored file contents) stay in the pack files until you rewrite history deliberately. This guide covers both, plus the CI (continuous integration), locking and quota behaviour that shows up a week later.

What you’ll accomplish

  • Install Git LFS and confirm the clean and smudge filters are wired into your Git config.
  • Track binary patterns in .gitattributes and verify that commits contain pointers, not blobs.
  • Decide honestly between tracking from now on and rewriting history with git lfs migrate import.
  • Clone and build in CI without paying for a full LFS download on every job.
  • Use git lfs ls-files, git lfs prune, and file locking to keep the working setup healthy.

Prerequisites

  • Git 2.11 or newer. That release introduced the long-running filter-process protocol Git LFS uses; older Git spawns one process per file and crawls on large trees.
  • The git-lfs binary, a separate program and not part of Git. apt install git-lfs on Debian and Ubuntu, dnf install git-lfs on Fedora and the RHEL family, brew install git-lfs on macOS. Git for Windows bundles it. Confirm with git lfs version; 3.x covers everything here.
  • A remote whose host runs an LFS server. GitHub, GitLab, Bitbucket, Gitea, and Forgejo all do. A bare repository on a plain SSH box does not, unless lfs.url points at something that speaks the LFS batch API.
  • Write access, plus the ability to force-push if you reach the history-rewriting section.
  • Honest version caveat: git lfs migrate arrived in 2.2.0 and its --everything flag later, so a distribution shipping 2.0 rejects the migration commands below. Check before planning a migration window.

How it works

Git has a filter mechanism that predates LFS. When a path matches an attribute with filter=lfs, Git runs content through the clean filter on the way into the object store and the smudge filter on the way out into the working tree. git lfs install writes those definitions into your Git config, which is why installing it once per machine matters more than anything you do inside a repository.

Data flow diagram showing the same file path holding real binary content in the working tree, a small UTF-8 pointer file in the Git commit, and a content-addressed blob on the LFS server, with git add running the clean filter, git checkout running the smudge filter, git push uploading the blob, and git lfs pull fetching it back, plus a panel showing the three lines of a real pointer file

The clean filter reads the file, computes its SHA-256, writes the content into .git/lfs/objects/, and hands Git a three-line text file instead: a spec version URL, oid sha256: plus the digest, and size in bytes. That text is what gets committed, what git log -p shows, what git diff compares, and what a merge sees. The smudge filter runs the trade in reverse on checkout, downloading the object from the server first if it is not in the local cache.

This explains the classic symptom. Someone clones on a machine where git-lfs is not installed, so no filters exist, so Git checks out exactly what is in the commit: 130-byte text files starting with version https://git-lfs.github.com/spec/v1. Nothing is broken and nothing is lost — the pointer is doing its job, and the person needs the tool that resolves it.

The transfer is separate from the Git transfer. git push sends commits over the normal protocol and, in the same command, uploads the LFS objects those commits reference over HTTPS to the batch API. That is why a push can hang on Uploading LFS objects while the Git side is already finished.

Because the filters key off .gitattributes, that file is the entire configuration surface. It is versioned like any other file, which is the point: check it in and every clone gets the same rules. Forget to commit it and your machine writes pointers into a branch where everyone else writes 240 MB blobs.

Step-by-step

  1. Install the filters on the machine. This is a per-user, per-machine action that edits ~/.gitconfig. Run it once; running it again is harmless.

    git lfs install

    Expected: Updated Git hooks. followed by Git LFS initialized. when run inside a repository, or just the second line outside one. Confirm the filter is actually registered:

    git config --get filter.lfs.process

    Expected: git-lfs filter-process. If this prints nothing, no amount of tracking will work. Use git lfs install --local to confine the filters to one repository instead of your global config.

  2. Track the patterns you care about. git lfs track scans nothing and moves nothing; it appends a line to .gitattributes. Quote the patterns so your shell does not expand them.

    git lfs track "*.psd"
    git lfs track "*.mp4"
    git lfs track "assets/textures/**"

    Expected: Tracking "*.psd" and so on, one line per call. Check the result:

    cat .gitattributes

    Expected:

    *.psd filter=lfs diff=lfs merge=lfs -text
    *.mp4 filter=lfs diff=lfs merge=lfs -text
    assets/textures/** filter=lfs diff=lfs merge=lfs -text

    The trailing -text disables newline conversion, which would corrupt a binary. git lfs track with no arguments lists the patterns in effect and which file declared each one.

  3. Commit .gitattributes first, on its own. This is the step people skip. Until it is in a commit, tracking exists only on your disk.

    git add .gitattributes
    git commit -m "Track binary assets with Git LFS"

    Expected: a commit touching one file. Keeping it separate makes the diff obvious in review and lands the rule before any binary does.

  4. Add a large file and check what Git recorded. Do not assume it worked — read the object.

    git add assets/intro.mp4
    git commit -m "Add intro video"
    git cat-file -p HEAD:assets/intro.mp4

    Expected: three lines of text, not binary garbage.

    version https://git-lfs.github.com/spec/v1
    oid sha256:4d7a21c9b0e35f8ac41d2e7b6590fa38c7d41e0b9a52c6f83d1749ee0b2c53f0
    size 251658240

    If git cat-file floods your terminal with binary, the file was committed as a normal blob — see the troubleshooting note on renormalising.

  5. Push and watch both transfers. The LFS upload runs before the Git ref update.

    git push origin main

    Expected output leads with the LFS phase, then the tiny Git transfer:

    Uploading LFS objects: 100% (1/1), 240 MB | 11 MB/s, done.
    Enumerating objects: 5, done.
    Writing objects: 100% (3/3), 412 bytes | 412.00 KiB/s, done.
    To github.com:example/media.git
       3f9a1c2..7b40e18  main -> main
  6. Deal with what is already in history. New tracking rules apply to new commits only. Everything committed earlier is still a full blob in the pack files, and every clone still downloads it.

    Side-by-side comparison of a four-commit chain where only the newest commit stores a pointer and the pack files still hold the old blobs, versus the same chain after git lfs migrate import where every commit stores a pointer, the pack files shrink, but every commit SHA changes and a force-push is required

    Start by measuring, which changes nothing:

    git lfs migrate info --everything --above=10MB

    Expected: a table of extensions with total size and file counts:

    migrate: Sorting commits: ..., done.
    migrate: Examining commits: 100% (48/48), done.
    *.mp4   1.4 GB   6/6 files(s)  100%
    *.psd   820 MB  12/12 files(s)  100%

    If those numbers justify the disruption, rewrite:

    git lfs migrate import --everything --include="*.mp4,*.psd"

    Expected: migrate: Rewriting commits: 100% (48/48), done. then a list of updated refs. Every commit that touched a matching path now has a new SHA — the same mechanics as our guide to interactive rebase, applied to the whole graph.

    Be blunt with your team about the consequences. Open pull requests against rewritten branches break, every existing clone diverges and must be re-cloned rather than merged, and anything pinned to an old SHA stops resolving. Freeze the branch, run the migration, force-push, and say so in one message:

    git push --force-with-lease origin --all
    git push --force-with-lease origin --tags

    Expected: + 3f9a1c2...7b40e18 main -> main (forced update). If you cannot force-push — protected branches, an active release, a regulated audit trail — use git lfs migrate import --no-rewrite instead. It converts the current files to LFS in one new commit and leaves history alone. The repository does not shrink, but the tip stops carrying blobs forward.

  7. Make CI clones cheap. A build job that only needs source code should not download gigabytes of textures. GIT_LFS_SKIP_SMUDGE=1 tells the smudge filter to write the pointer and skip the download.

    GIT_LFS_SKIP_SMUDGE=1 git clone git@github.com:example/media.git

    Expected: the clone finishes at Git speed with no Downloading LFS objects phase, and the working tree contains pointers. Then fetch only what the job needs:

    cd media
    git lfs pull --include="assets/textures/**" --exclude=""

    Expected: Downloading LFS objects: 100% (14/14), 180 MB | 22 MB/s, done. and only those paths materialised. The empty --exclude="" overrides any lfs.fetchexclude already in config. On GitHub Actions, actions/checkout takes an lfs: true input that does the full download — leave it off and run the targeted git lfs pull yourself.

  8. Lock files that cannot be merged. Two people editing the same .psd produce a conflict no merge tool can resolve. Locking surfaces the collision before the work is done.

    git lfs track "*.psd" --lockable
    git lfs lock design/hero.psd
    git lfs locks

    Expected: Locked design/hero.psd, then a listing like design/hero.psd alice ID:12. The --lockable flag lands in .gitattributes and makes matching files read-only in the working tree until locked, which is the part that actually stops accidental edits. Releasing a lock is the inverse subcommand, and git lfs locks --verify is what a pre-push hook uses to reject a push that would clobber a lock someone else holds. Locking needs server support; GitHub, GitLab, and Gitea implement it, others vary.

  9. Reclaim local disk. .git/lfs/objects/ accumulates every version you ever checked out. git lfs prune deletes local copies that are old and confirmed present on the server.

    git lfs prune --dry-run --verbose

    Expected: a count of candidate objects and the space they occupy, with nothing deleted. Run it without --dry-run when the list looks right. Retention is governed by lfs.pruneoffsetdays and lfs.fetchrecentrefsdays; the defaults keep recent branches usable offline. Pruning never touches the server, so it cannot cost anyone else data.

Validate it works

The only test that means anything is a fresh clone in a scratch directory, because that is what a new teammate and every CI job do.

cd $(mktemp -d)
git clone git@github.com:example/media.git
cd media
ls -l assets/intro.mp4

Expected: a file of real size — 251658240 bytes, not 130. If it is 130 bytes, the smudge filter did not run on this machine.

Confirm Git and LFS agree on which files are managed:

git lfs ls-files

Expected: one line per tracked file — short oid, a flag character, then the path:

4d7a21c9b0 * assets/intro.mp4
9c1f0ab472 * design/hero.psd

The * means the object is present locally and the working file is real content. A - means only the pointer is there — correct after a skip-smudge clone, wrong after a normal one.

Check the client’s view of the configuration, including which endpoint it will talk to:

git lfs env

Expected: an Endpoint= line with your remote’s LFS URL, the local media directory path, and the filter.lfs.* values. A wrong or empty endpoint explains most authentication failures.

If you migrated, prove the repository actually got smaller:

git count-objects -vH

Expected: size-pack in the tens of megabytes rather than gigabytes, compared against the figure you recorded before the rewrite. Run git lfs fsck too — it verifies that every pointer in the checkout has an object whose contents hash to the recorded oid.

Troubleshooting

  • A teammate’s clone is full of 130-byte text files. git-lfs is not installed on that machine, or it is installed but git lfs install was never run, so no filters exist in their config. Have them install the binary, run git lfs install, then git lfs pull in the existing clone to replace pointers with real content — no re-clone needed. If the tool is present and configured, check whether GIT_LFS_SKIP_SMUDGE is exported in their shell profile; git config --get filter.lfs.smudge shows git-lfs smudge --skip %f if --skip-smudge was ever installed.
  • Encountered 1 file(s) that should have been pointers, but weren't. A matching file was committed before the pattern was tracked, so the blob is real content while .gitattributes claims otherwise. Run git add --renormalize . and commit; that re-runs the clean filter over everything the attributes now cover. It fixes the tip only — the old blob is still in history, which is what step 6 exists for.
  • This repository is configured for Git LFS but 'git-lfs' was not found on your path. The hooks installed by git lfs install are running but the binary is gone — common after a PATH change, a Homebrew cleanup, or in a container image that copied .git/ but not the tool. Install git-lfs, or remove the hooks with git lfs uninstall and accept that you are handling pointers.
  • batch response: This repository is over its data quota. The host refused the transfer, not Git. Every major provider meters LFS storage and download bandwidth separately from repository size, and a CI pipeline that clones with LFS on every run is usually the largest consumer by a wide margin. Cache .git/lfs/objects/ between jobs, switch jobs that do not need assets to GIT_LFS_SKIP_SMUDGE=1, and run git lfs prune on build agents. If storage rather than bandwidth is the constraint, remember that deleting a file from the tip frees nothing — history still references the object.
  • error: external filter 'git-lfs filter-process' failed during checkout. Usually authentication: the smudge filter needs credentials for the LFS endpoint independently of the SSH key that fetched the commits. Run git lfs env to see the endpoint, then git lfs pull on its own to get the real error message, which the filter swallows. Behind a self-hosted reverse proxy, also check the request body size cap, which fails only on large objects.

When this is the wrong approach

Git LFS is for source assets that belong to a commit — the layered PSD a designer edits, the reference video a test compares against, the texture that ships with the level. It is a poor fit for build outputs. Compiled binaries, container images, and release tarballs belong in an artifact registry or object storage, where retention, immutability, and promotion between environments are first-class instead of bolted onto version control.

It is also the wrong tool when the real problem is a proper asset pipeline. If you are versioning 40 GB of raw footage that three people ever touch, a shared storage volume with a naming convention and a manifest checked into Git serves better than making every clone depend on it. And check what your host meters first: several providers bill LFS bandwidth on every download, turning a per-commit CI clone into a recurring cost unrelated to repository size. Self-hosting removes the meter but adds a service to back up — lose it and every pointer in history resolves to nothing.

LFS is also not the only lever for a large repository. If the weight is thousands of source files rather than a few huge binaries, partial clone and sparse checkout address that directly and compose with LFS rather than competing with it. The authoritative references are git-lfs.com and the git-lfs repository, whose docs/ directory carries the man pages for every subcommand and the pointer file specification.

If this is the kind of work you’d rather never own again, it’s the daily job of Managed Cloud Ops — book an infrastructure review.