Draft:HippoML

HippoML, Inc.
TypePrivate (acquired)
IndustryArtificial intelligence, Machine learning, Computer software
Founded2023
FoundersBing Xu, Hao Lu, Terry Chen
Defunct2024
HeadquartersBellevue, Washington, United States
Key people
Bing Xu (CEO)
Terry Chen (VP of Engineering)
Hao Lu
ProductsHippoEngine, PrivateCanvas
ParentNVIDIA

HippoML, Inc. was an American artificial intelligence software company that developed inference engines and optimization tools for generative AI models.[1] Founded in 2023 and based in Bellevue, Washington, the company focused on improving deployment efficiency for large language models and other AI workloads across NVIDIA, AMD, and Apple Silicon hardware.[1][2][3] In 2024, HippoML was acquired by NVIDIA.[4]

History

HippoML was founded in Bellevue, Washington, in 2023 by Bing Xu, Hao Lu, and Terry Chen.[1][5][6][7] Before founding the company, Xu and Lu had worked on deep learning systems including AITemplate, while Chen had also contributed to GPU optimization frameworks including AITemplate.[5][6][7]

During its independent operation, HippoML published engineering results on model serving, attention optimization, quantized inference, and local AI applications.[2][8][9][10][3] In 2024, the company was acquired by NVIDIA.[4]

Technology and products

HippoML's primary technology was HippoEngine, a GPU inference engine designed to compile machine learning models ahead of time into standalone binary code rather than depend on large runtime environments. HippoML said the engine used a unified runtime and API with support for NVIDIA CUDA, AMD ROCm, and Apple Metal.[2]

HippoML also described a multiple layers model caching system for managing model weights across NVMe SSDs, host memory, and GPU memory. According to the company, this reduced model activation times to about 450 milliseconds.[2]

The company published several blog posts about transformer attention optimization, including variable-length attention, Apple Silicon acceleration, 8-bit HippoAttention, and claims of achieving 1 petaFLOPS attention performance on a single NVIDIA H100 SXM GPU.[8][9][10]

To demonstrate its inference stack on consumer hardware, HippoML released PrivateCanvas, a local desktop AI application for Windows, Linux, and macOS. HippoML described it as an offline tool for image generation and editing that bundled models such as Stable Diffusion XL, Segment Anything, and other generative models into one environment.[3]

HippoML also wrote about decentralized AI inference and the use of local devices for AI workloads in a February 2024 blog post.[11]

Post-acquisition

Following the acquisition, former HippoML founders and engineers appeared as authors on NVIDIA Technical Blog posts related to AI inference, including posts on automated GPU kernel generation and DeepSeek-R1 inference performance in 2025.[5][6][7][12][13]

See also

References

  1. ^ a b c "HippoML company profile". PitchBook. Retrieved March 30, 2026.
  2. ^ a b c d "Unified DataCenter & Local Foundation Model Serving: Beyond Docker Way". HippoML Blog. January 8, 2024. Retrieved March 30, 2026.
  3. ^ a b c "Super AI Creativity App Run with Local GPU on Mac/Windows/Linux [Early Access]". HippoML Blog. January 2, 2024. Retrieved March 30, 2026.
  4. ^ a b "Zheng Zhou". Wilson Sonsini. Retrieved March 30, 2026.
  5. ^ a b c "Bing Xu". NVIDIA Technical Blog. Retrieved March 30, 2026.
  6. ^ a b c "Hao Lu". NVIDIA Technical Blog. Retrieved March 30, 2026.
  7. ^ a b c "Terry Chen". NVIDIA Technical Blog. Retrieved March 30, 2026.
  8. ^ a b "Up to 80X Speedup in Multi-Head Attention on Apple Silicon". HippoML Blog. December 11, 2023. Retrieved March 30, 2026.
  9. ^ a b "8bit HippoAttention: Up to 3X Faster Compared to FlashAttentionV2". HippoML Blog. January 17, 2024. Retrieved March 30, 2026.
  10. ^ a b "PetaFLOPS Inference Era: 1 PFLOPS Attention, and Preliminary End-to-End Results". HippoML Blog. February 7, 2024. Retrieved March 30, 2026.
  11. ^ "Vision Pro, Decentralized GenAI and AI PC". HippoML Blog. February 7, 2024. Retrieved March 30, 2026.
  12. ^ "Automating GPU Kernel Generation with DeepSeek-R1 and Inference Time Scaling". NVIDIA Technical Blog. February 12, 2025. Retrieved March 30, 2026.
  13. ^ "NVIDIA Blackwell Delivers World-Record DeepSeek-R1 Inference Performance". NVIDIA Technical Blog. March 18, 2025. Retrieved March 30, 2026.

Content Disclaimer

Informasi ini disarikan dari Wikipedia dan disajikan kembali untuk tujuan edukasi. Konten tersedia di bawah lisensi CC BY-SA 3.0. Kami tidak bertanggung jawab atas ketidakakuratan data yang bersumber dari kontribusi publik tersebut.

  1. The information displayed on this website is sourced in part or in whole from Wikipedia and has been adapted for the purpose of restating it. We strive to provide accurate and relevant information, however:
  2. There is no guarantee of absolute accuracy. Wikipedia is an open, collaborative project that can be edited by anyone, so information is subject to change.
  3. It is not intended to constitute professional advice. The content displayed is for informational and educational purposes only. For important decisions (e.g., medical, legal, or financial), please consult a professional.
  4. Content copyright. Wikipedia is licensed under the Creative Commons Attribution-ShareAlike License (CC BY-SA). This means that content may be reused with appropriate attribution and shared under a similar license.
  5. Responsible use. Any risk arising from the use of information from this website is entirely the responsibility of the user.