Blog - AI

Adapter for fine tuning LLM
OSS

Creating a patch for integrating adapter modules into the RoBERTa model allows for efficient fine-tuning tailored to specific tasks. By embedding lightweight adapters within the pre-trained layers of RoBERTa, we can freeze the majority of the original parameters, updating only the new adapter parameters. This approach significantly reduces computational overhead and speeds up training. Implementing this patch involves defining the adapter architecture, integrating it within RoBERTa, and focusing the training on these adapters to leverage pre-existing knowledge effectively. Don't forget to check the source code!

pytorch AI

Closing a Chapter, Opening a New One

Reflecting on my journey as an AI Software Engineer at Solico Group and announcing my next step as an AI Researcher at Monash University.

AI Career Research Machine Learning LLM RAG

Building a Scalable Data Pipeline for LLM Training: From Streaming to Production

A deep dive into creating an enterprise-grade data collection and processing pipeline for Large Language Model training, featuring async processing, quality control, and tokenization at scale.

AI Data Engineering LLM Pipeline Machine Learning Python AsyncIO

Adapter for fine tuning LLM OSS

Closing a Chapter, Opening a New One

Building a Scalable Data Pipeline for LLM Training: From Streaming to Production

Adapter for fine tuning LLM
OSS