Deep Learning Compiler & Frameworks @Intel | AI/LLM Middleware pytorch engineer@habana labs| former-Research@Morgan Stanley ,@Imperial College London,@HSBC | Author | PyData & ISO:C++ Conferences | Google-research | NIT
There is a growing demand on bare metal performance for LLMs which can be made to run on low end devices. Particularly from compiler benchmarking, applications such as llama.cpp ,whisper.cpp and many others are extremely important. With great collaboration from Georgi Gerganov and Intel Corporation, we have enabled SYCL runtime for llama.cpp and whisper.cpp. This is one of the major applications which use #oneapi and #SYCL runtime (Intel® oneAPI DPC++ Library) for efficient performance across different vendor compiler runtimes. Intel Software #compiler #sycl #llms #oneapi #genai #clang
Piotr Malicki
9mo
That's an incredible achievement! Keep pushing the boundaries of performance! 💻
Founder & CEO, Group 8 Security Solutions Inc. DBA Machine Learning Intelligence
9moThanks a lot for posting!