The Death of Software Engineering Is Premature

There is a lot to be excited about when it comes to advancement in language models.  The people that benefit the most are software engineers because it can enhance the productivity of knowledgeable engineers. 

While a simple prompt such as "build me a web app with a conversation interface," will result in a functional web app (this is great because building a basic commodity application is now accessible to everyone),  these aren't the type of applications software engineers are normally tasked to write.  Software engineers are tasked with building multi-layered and evolving software that can't be fully described with a few prompts.  To build the software that businesses need requires skilled and knowledgeable people to direct the engineering work of the AI.

I built a simple app to filter news headlines that is small enough to be digestible in a post that I think shows how far a layman can go with building software using LLMs and why software engineers are still needed.  

The First Prompt

Let's start with what many executives might think makes software engineers unnecessary: 

Build me an app that will filter out political articles from Google News.

ChatGPT understood enough to generate an app using keywords to filter the headlines.  One time it created a React app and a second time it used Python.  Both times it used newsapi.org to get the news feeds and require you to understand how to build and run the app.  The main issue is that the app isn't really what I wanted.  It provides news search which the results are then matched with keywords to decide what to filter out.  I wanted the news that I normally see when I visit news.google.com minus the political articles so I tell ChatGPT precisely that:

I don't want to search for news.  I want to see the articles that I see when visiting news.google.com minus political articles

The first time I asked ChatGPT it understood this enough to switch to using the Google News RSS feed which is excellent\!  The second time, it concluded that the way is to scrape news.google.com.  Both of these prompts highlight that some specialized knowledge is needed.  Does the programming language matter?  Should it use RSS or web scraping?  How do you run the code it generates?  Can these questions be ignored? Who in the organization does the CEO expect to be able to answer these questions?

The Second Prompt

While the CEO might not be able to use the AI, it is possible that a non-engineer or someone who doesn't know how to program can know enough to give more details in the prompt that improves on the first prompt.  A technical product manager could give the AI the extra details: 

1. single page web app using material design   
1. have a header with the company logo on the left  
2. have a settings menu on the right side of the header  
3. ...  
2. web app will get the list of headlines to display   
3. pull headlines from RSS feeds and have a LLM return the items that are not political  
4. build it with language X on the backend and basic Javascript and CSS on the front end.

(4) was included to demonstrate how software often needs to fit in with a business' existing infrastructure.  If the AI returns a React app but there is no infrastructure to build and deploy React apps then we'd be looking at additional costs and efforts to add that ability simply because the AI had no knowledge of what is reasonable for a specific business.

LLMs are capable of generating this app although the generated code didn't actually work and I had to guide it to how to fix things but for now let's assume that the LLM generated a fully working app. 

If we were to stop here then we might conclude that software engineers aren't needed anymore, but I believe the conclusion is actually that for companies that previously did not have engineers that they now have access to some skills their organization never had before.  For example, a small company who has an office manager built a spreadsheet that they're running their business on would benefit because while their spreadsheet worked it is still limited.  The next step to expand the spreadsheet that would previously require some coding knowledge to extend the worksheet can now be done by the office manager with the aid of an AI.  There is cost saving for the company because they didn't have to hire a consultant (these companies probably don't have enough work to hire a full time engineer to be on staff).

Remember that I left out a number of things that the AI cannot do including turning the code into an app and deploying the app, but the biggest hurdle would be if the AI did not meet all the requirements for the app initially.  I had success prompting the AI to make certain changes and enhancements but multiple times the AI would make a change that didn't work and go further-and-further down the rabbit hole and no prompting got it to fix the problem until I have it very specific instructions on how to fix ("In function X, change the max/min values to be Y for variable V").  

Basically, maintaining, fixing and enhancing the app is where the challenge is and that's where most engineers are spending their time.

The Third Prompt

Prompt 3 (in real life was actually what I did first) was the fastest and most productive way towards building a working application and this was to have a design and give the AI many specific implementation instructions.  I knew the architecture, algorithm and code structure and used that knowledge to guide the AI so that it was essentially very fast typist of code:

• Write a FetchRSS function that takes in a string value for the RSS feed configuration file path.  
• In FetchRSS, open up the configuration file and loop through each line to get the URL.  
• For each URL, fetch it and parse the RSS response into a slice of strings  
• Oh, ignore any lines that are blank or starts with '\#'  
• ...

Since I enjoyed typing out code, I tended to write my own stuff but for the "boring" codes (e.g. handling when err \!= nil) I'll have the AI write it.

I was able to complete the code much faster, make fewer trips to look up references and documentation and with few bugs from typos.

Here's the problem, though.  While AI is capable of generating valid code, it isn't "working" code.  It isn't code that can be used directly in a business' infrastructure.  AI is still struggling to understand an entire code base and writing code that works within a business requires also understanding the environment the code is running in.  If the code will be run in complete isolation then it might be able to run but even a simple function such as "GetUserName" depends on where the user name is stored?  What integration must the AI be aware of in order to get the user name?  In a real environment, the AI simply just gives code snippets that the engineer must still adapt into the organization's infrastructure.

Conclusion

Having an AI capable of building software for your business is not realistic.  My example app is too simplistic to be any company's product and it still wasn't able to do that.  

Having a knowledgeable person on staff using AI will increase your staff's ability to do their jobs better and allow companies to do things they previously couldn't but most of these things are not things software engineers typically do.   If companies try to have non-engineers do software engineering work with the AI will likely result in decreased productivity.  Any gains from having something done quickly at the beginning will also be quickly overshadowed by the inability to maintain and enhance the software.

It is ultimately a decision companies have to make for themselves.  Is a % reduction in compensating engineers a greater value then expanding by factors the productivity and capabilities of the engineers (reduce cost vs growth)? 

What is complex today will not be as complex tomorrow.  They will become common and AI will be able to take care of those things, but as AI takes over more of the common stuff that will allow engineers to tackle the next level of complexity with more originality because businesses will need it to survive.  There are leaders at technology companies boasting about being able to get rid of their software engineers or how they will stop hiring more engineers.   These leaders are making a choice to make cheap commodity products in exchange for growth and innovation, but they might find themselves racing to the bottom instead of accelerating to the top.  

Can AI someday outpace people?  Maybe.  But it's not now.  Declaring that engineers aren't needed is pre-mature.