v0.0.0beta26
·
119 commits
to main
since this release
What's Changed
- [SC-836587] Pin boto3 and urllib3 versions to fix error in inference image by @edgan8 in #432
- include stop string in completions output by @saiatmakuri in #435
- Logging post inference hook implementation by @tiffzhao5 in #428
- add codellama-70b models by @saiatmakuri in #436
- Add hook validation and support logging for python client by @tiffzhao5 in #437
- Azure refactor for async endpoints by @squeakymouse in #425
- Remove post inference hook handling in main container by @tiffzhao5 in #438
- Clean up logs for logging hook by @tiffzhao5 in #439
- Fix Infra Task Gateway by @saiatmakuri in #443
- support gemma models by @saiatmakuri in #444
- Fix infra config dependency by @squeakymouse in #449
- Add emitted timestamp for logging by @tiffzhao5 in #450
- Change cache update time for async endpoint integration test by @tiffzhao5 in #451
- Bump aiohttp from 3.9.1 to 3.9.2 in /model-engine by @dependabot in #446
- Bump python-multipart from 0.0.6 to 0.0.7 in /model-engine by @dependabot in #447
- Bump gitpython from 3.1.32 to 3.1.41 in /model-engine by @dependabot in #453
- Log endpoint in sensitive_log_mode by @squeakymouse in #455
- Bump orjson from 3.8.6 to 3.9.15 in /model-engine by @dependabot in #456
- Allow the load test script to use a csv of inputs by @seanshi-scale in #440
- add some debugging to vllm docker by @yunfeng-scale in #454
- Add product label validation by @edgan8 in #442
- Add log statement for gateway sending async task by @tiffzhao5 in #459
- Some batch inference improvements by @yunfeng-scale in #460
- Fix cacher by @yunfeng-scale in #462
- Fix vllm batch docker image by @yunfeng-scale in #463
- Add tool completion to batch inference by @yunfeng-scale in #461
- fix llm-engine finetune.create failures by @ian-scale in #464
- Change back batch infer GPU util and add tool completion client changes by @yunfeng-scale in #465
New Contributors
Full Changelog: v0.0.0beta25...v0.0.0beta26