by abacusai · Agent Tool · ★ 600
Extending LLM Context Length The choice of how to encode positional information for transformers has been one of the key components of LLM architectures. An area that has been interesting to us and others in the community recently is whether LLMs can be extended to longer contexts. We have conducted a range of experiments with different schemes for extending context length capabilities of Llama, which has been pretrained on 2048 context length with the RoPE (Rotary Position Embedding) encoding.
| Stars | 600 |
| Forks | 46 |
| Language | Python |
| Category | Agent Tool |
| License | Apache-2.0 |
| Quality Score | 52.9599012604374/100 |
| Open Issues | 7 |
| Last Updated | 2023-11-17 |
| Created | 2023-07-27 |
| Platforms | python |
| Est. Tokens | ~36k |
These tools work well together with Long-Context for enhanced workflows:
Explore other popular agent tool tools:
Long-Context is This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and benchmark tasks that evaluate a model’s information retrieval capabi. It is categorized as a Agent Tool with 600 GitHub stars.
Long-Context is primarily written in Python.
You can find installation instructions and usage details in the Long-Context GitHub repository at github.com/abacusai/Long-Context. The project has 600 stars and 46 forks, indicating an active community.
Long-Context is released under the Apache-2.0 license, making it free to use and modify according to the license terms.