LLM Based Automated Code Synthesis & Evaluation Pipeline

Python, Java, pytest, Large Language Models (LLM), JSON, Pandas, SciPy, Shell Scripting, Git, Prompt Engineering

• Developed an end-to-end automation pipeline using LLM's for generating programming problem datasets and selecting a reproducible subset for evaluation.
• Engineered systems to interact with large language models for both code synthesis and translation tasks, including experiments with advanced quantization configurations.
• Implemented automated prompting for detecting bugs and generating detailed code coverage reports to enhance code quality.
• Designed integrated validation routines to verify the integrity of generated datasets, model outputs, and essential project artifacts.
• Built command-line scripts and automated workflows to seamlessly orchestrate dataset preparation, model prompting, and result evaluation, ensuring reproducibility and reliability.
• Contributed to the project's impact by streamlining the process of code evaluation and translation, thus enabling robust experimental research and development in automated code synthesis.