Paper
Efficient Large Language Model Inference with Limited Memory
Tackles running LLMs that don’t fit DRAM by keeping weights in flash and streaming them efficiently. Introduces a cost model and techniques like windowing an...
Related papers
14
-
University of California, Berkeley
-
University of California, Santa Cruz
-
Tribhuvan University
-
University of Texas at Arlington
-
Harvard University
-
The University of Texas at Austin
Looking for more?
Open the arXiv / PDF link for the full paper. Abstract and authors appear here when enrichment has run.