WizardCoder: Empowering Large Pre-Trained Language Models to Follow Complex Instructions

Published in

AI Mind

4 min readAug 28, 2023

Introduction

The world of coding has been revolutionized by the advent of large language models (LLMs) like GPT-4, StarCoder, and Code LLama. However, the latest entrant in this space, WizardCoder, is taking things to a whole new level. WizardCoder is a specialized model that has been fine-tuned to follow complex coding instructions. It leverages the Evol-Instruct method to adapt to coding tasks, making it a powerful tool for developers.

The Big Announcement: WizardCoder-Python-34B-V1.0

The most recent release, WizardCoder-Python-34B-V1.0, has set new benchmarks by achieving a 73.2 pass@1 on the HumanEval Benchmarks, surpassing even GPT-4 and ChatGPT-3.5. This is a significant milestone in the field of AI and coding, as it opens up new possibilities for automating complex tasks.

How Can You Use WizardCoder?

Step 1: Installation

First, you’ll need to install the required packages. You can do this using pip:

pip install transformers deepspeed

Step 2: Importing the Model

You can import the WizardCoder model from Hugging Face’s model hub:

from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("WizardLM/WizardCoder-Python-34B-V1.0")
model = AutoModelForCausalLM.from_pretrained("WizardLM/WizardCoder-Python-34B-V1.0")

Step 3: Using the Model

Let’s say you want to generate a Python function to calculate Fibonacci numbers. You can do this as follows:

instruction = "Write a Python function to calculate the nth Fibonacci number."
input_ids = tokenizer(instruction, return_tensors="pt").input_ids
output = model.generate(input_ids)
decoded_output = tokenizer.decode(output[0], skip_special_tokens=True)
print(decoded_output)

This will output a Python function that you can then use in your projects.

What Kind of Technologies Can Be Generated?

The 34B model is not just a coding assistant; it’s a powerhouse capable of:

Automating DevOps Scripts: Generate shell scripts or Python scripts for automating tasks.
Data Analysis: Generate Python code for data preprocessing, analysis, and visualization.
Machine Learning Pipelines: Generate end-to-end ML pipelines, from data collection to model deployment.
Web Scraping: Generate code for web scraping tasks.
API Development: Generate boilerplate code for RESTful APIs.
Blockchain: Generate smart contracts for Ethereum or other blockchain platforms.

The Power of DemoGPT

When talking about powerful agents on GitHub, it’s impossible not to mention DemoGPT. DemoGPT is one of the most powerful agents on GitHub for several reasons:

Extensive Library: It has a rich set of functionalities that can be extended easily.
Community Support: A large community of developers actively contributes to its ecosystem.
Scalability: Designed to scale, it can handle a wide range of tasks, from simple to complex.
Marketplace: DemoGPT is soon launching its marketplace in collaboration with LangChain, where all these generated applications will be shared.

Future Plans: Code LLama and DemoGPT

DemoGPT has plans to support Code LLama as a locally run model in the near future. This will make it even more powerful and versatile, offering users the ability to run large language models on their local machines without the need for an internet connection.

Conclusion

WizardCoder is a groundbreaking development in the field of AI and coding. Its recent achievements on the HumanEval Benchmarks are a testament to its capabilities. With the upcoming marketplace from DemoGPT and LangChain, the possibilities are endless. So, whether you are a seasoned developer or a newbie, WizardCoder is a tool you definitely want in your arsenal.

Citation

If you find WizardCoder useful, please consider citing the original paper:

@misc{luo2023wizardcoder,
      title={WizardCoder: Empowering Code Large Language Models with Evol-Instruct}, 
      author={Ziyang Luo and Can Xu and Pu Zhao and Qingfeng Sun and Xiubo Geng and Wenxiang Hu and Chongyang Tao and Jing Ma and Qingwei Lin and Daxin Jiang},
      year={2023},
      eprint={2306.08568},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Disclaimer

The content produced by WizardCoder is influenced by various factors, including randomness. Therefore, the accuracy of the output cannot be guaranteed. Always review and test the generated code before using it in production environments.

Feel free to share this guide and let us know your thoughts! Happy coding! 🚀