🔄Continue
Continue is a VS Code or Intellij IDEA plugin which integrates AI code assistant to your IDE and allows you to use both local models and enterprise (paid) models. I tried it with local Ollama and paid ChatGPT-4o and Claude 3.5.
🤯My first WOW moment was when I understood that you need 3 models. One for chat, one for code completion (
tabAutocompleteModel) and one for indexing your large code base and store embeddings (embeddingsProvider). For all three models there are variants which you can host yourself and they work quite good.📉 The tool configuration and UI is far from one where our grandparents will be able to use it. You still need to be techy and more or less understand what's going on. I guess we are on early stages of the development of the market.
Overall I liked it and recommend to try it in case you don't want to pay for GitHub Copilot (where integration and configuration is much simpler)
📊 AI code assistant model assessment
I tried to understand which AI code assistant model is the best. It was not easy to find a reliable source, since I was afraid that half of articles were written by AI.
There is recognised Code Generation evaluation framework called HumanEval. It was established by OpenAI and it contains some tasks, which model shall solve using Python. Many models are run against this framework and this way community understands what is better. So far it's ChatGPT-4o.
Also there is a nice website where models are visually compared.
❓If the model is able to solve certain tasks in
Python better than the other model, are we able to say that it will generate Java code better? I doubt.▪️This allows me to conclude, that at this stage there is no silver bullet — either one model or one tool which will make you x100 engineer. But it's safe to invest in the tools which bring you additional benefits and allow you to still be flexible. For example, Continue, where you will be able to switch between models and use those which bring benefit base on your particular needs.