Skip to content

feat: expose progress_callback in Llama wrapper#2340

Open
Niels-BW wants to merge 1 commit into
abetlen:mainfrom
Niels-BW:feat/expose_progress_callback
Open

feat: expose progress_callback in Llama wrapper#2340
Niels-BW wants to merge 1 commit into
abetlen:mainfrom
Niels-BW:feat/expose_progress_callback

Conversation

@Niels-BW

Copy link
Copy Markdown

Exposes progress_callback and progress_callback_user_data through the high-level Llama constructor.

llama_model_params already supports these fields, and the ctype bindings already expose them, but they are not currently available through the Llama API.

This allows Python applications to receive model loading progress and optionally abort model loading by returning False from the callback, consistent with the underlying llama.cpp API.

Implementation

  • Add progress_callback and progress_callback_user_data to the Llama constructor
  • Wrap the supplied Python callback with llama_progress_callback
  • Store the wrapped callback on the Llama instance to prevent it from being garbage collected before model loading completes.

Testing

This change was tested by successfully receiving model loading progress callbacks and aborting model loading by returning False from the callback.

@Niels-BW Niels-BW marked this pull request as ready for review July 12, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant