Cosign Genie: A Revolutionary Leap in Autonomous Software Engineering
–
Cosign has unveiled Genie, a new AI model fine-tuned from GPT-4. Genie can solve software engineering problems with a 43.8% success rate on SW Bench, a verified benchmark for software tasks. This shows a significant leap from the previous high score of 38.8% held by Amazon Q's developer agent.
Ally, co-founder and CEO of Cosign, noted that Genie uses human reasoning techniques. They trained Genie with real software engineering tasks, mimicking human problem-solving. Instead of random code generation, Genie thinks and plans like a human.
In a demo, Genie solved a GitHub issue in just 84 seconds. It fetched relevant files, wrote and tested code, and created a pull request. Genie can also understand comments and make changes as if it were a human colleague.
Ally explained that Genie’s iterative process sets it apart. Genie writes, tests, and revises code repeatedly until it solves the problem. This mimics how human engineers work, giving Genie a deep understanding of problem-solving.
One reason for Genie’s success is its training method. Cosign used a large dataset of human solutions, capturing the step-by-step decisions engineers make. Genie has seen more coding examples than any human could in a lifetime.
Cosign also used self-improvement in training. They had Genie solve coding problems and learn from its mistakes. Each new version of Genie was better, needing fewer corrections. This iterative learning loop boosted Genie’s performance significantly.
Future plans for Genie include expanding its capabilities and training it in more programming languages and frameworks. Cosign aims to create different-sized AI models for various tasks. They also plan to open-source a version of Genie to foster innovation in the AI community.
A key feature for businesses is Genie’s ability to understand specific codebases, even in uncommon programming languages. This means Genie can become an expert in a company’s unique coding style, making it a valuable tool for software development.
The rapid progress in AI models like Genie shows the potential for even more advanced systems in the future. With continuous improvements and new training methods, the software development space is evolving swiftly.
