Traster 2 days ago

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations of simple code. I don't want X to be abstracted this way, I want that...." etc. And then when I give it feedback 2,000 lines of code suddenly switch to 700 lines of completely different code and I can't keep up. And I don't want my codebase full of disjoint scripts that I don't really understand and all have weirdly different approaches to the problem. I want an AI that I have similar opinions to, which is obviously tough. It's like working with someone on their first day.

I don't know if it's giving the tools less self-confidence per se, but I think it's exposing more the design process. Like ideally you want your designer to go "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here" and we can approve that first, rather than going straight from prompt -> implementation.

19
34679 2 days ago

Just like with human engineers, you need to start with a planning session. This involves a back and forth discussion to hammer out the details before writing any code. I start off as vague as possible to see if the LLM recommends anything I hadn't thought of, then get more detailed as I go. When I'm satisfied, I have it create 2 documents, initialprompt.txt and TODO.md. The initial prompt file includes a summary of the project along with instructions to read the to do file and mark each step as complete after finishing it.

This ensures the LLM has a complete understanding of the overall goals, along with a step by step list of tasks to get there. It also allows me to quickly get the LLM back up to speed when I need to start a new conversation due to context limits.

globnomulous 2 days ago

In essence, I need to schedule a meeting with the LLM and 'hammer out a game plan.' Gotta make sure we're 'in sync' and everybody's 'on the same page.'

Meeting-based programming. No wonder management loves it and thinks it should be the future.

woah 2 days ago

LLMs are stealing the jobs of developers who go off half-cocked and spend three days writing 2000 lines of code implementing the wrong feature instead of attending a 30 minute meeting

hnthrow90348765 2 days ago

That's dumb, of course, but sometimes people really just do the bare minimum to describe what they want and they can only think clearly once there's something in front of them. The 2000 lines there should be considered a POC, even at 2000 lines.

devmor 2 days ago

and the jobs of developers that want to schedule another breakout session to discuss the pros and cons of a 2-line change.

ge96 2 days ago

Yeah... I'm gonna need to circle back on that

dowager_dan99 2 days ago

my manager has been experimenting have AI first right the specs as architecture decision records (ADR), then explain how the would implement them, then slowly actually implementing with lots of breaks, review and approval/feedback. He says it's been far superior to typically agent coding but not perfect.

ben_w 2 days ago

Meetings are how managers keep everyone else aligned with their goals.

apwell23 2 days ago

> This ensures the LLM has a complete understanding of the overall goals

Forget about overall goal. I have this simple instruction that i send on every request

"stop after every failing unit test and discuss implementation with me before writing source code "

but it only does that about 7 times out of 10. Other times it just proceeds with implementation anyways.

avandekleut 2 days ago

Ive found similar behaviour with stopping at linting errors. I wonder if my instructions are conflicting with the agent system prompt.

kenfox 2 days ago

System prompts themselves have many contradictions. I remember hearing an Anthropic engineer (possibly Lex Fridman's interview with Amanda Askell) talking about using exaggerated language like "NEVER" just to steer Claude to rarely do something.

apwell23 2 days ago

that doesn't work ( atleast not anymore)

jyounker 2 days ago

So it behaves just like a person.

apwell23 1 day ago

thats why we replaced people with machines . so we can have some predictability.

tharkun__ 1 day ago

Keyword: Some

apwell23 1 day ago

humans don't ignore an instruction 4 times out of 10 unless they have a reason to do it on purpose.

tharkun__ 23 hours ago

I congratulate you in that you only work with humans that never misunderstand, never forget a step in a long process they think they know by heart etc.

I guess you also think that we should get rid of checklists for pilots because they would never ignore an instruction they were clearly given during training except on purpose?

apwell23 10 hours ago

> I guess you also think that we should get rid of checklists for pilots because they would never ignore an instruction they were clearly given during training except on purpose?

Pilots ignore items in checklist 4 times out of 10? wtf

jappgar 1 day ago

Sadly this just doesn't pan out in larger more complex projects. It will write an implementation plan, not follow it, then lie and say it did.

SparkyMcUnicorn 23 hours ago

What tool and/or model are you calling "it"?

I'm using Claude Code on a large legacy monstrosity, and don't have this problem. There are problems and my flow automatically has it reviewing its own work in phased implementations, but even in the worst situations it's easy to get back on track.

dotancohen 1 day ago

  > I have it create 2 documents, initialprompt.txt and TODO.md.
That is an amazing approach. Which (AI) tools are you using?

iamkoch 2 days ago

This absolutely captures my experience.

My successful AI written projects are those where I care solely on the output and have little to no knowledge about the subject matter.

When I try to walk an agent through creating anything about which I have a deeply held opinion of what good looks like, I end up frustrated and abandoning the project.

I've enjoyed using roo code's architect function to document an agreed upon approach, then been delighted and frustrated in equal measure by the implementation of code mode.

On revelation is to always start new tasks and avoid continuing large conversations, because I would typically tackle any problem myself in smaller steps with verifiable outputs, whereas I tend to pose the entire problem space to the agent which it invariably fails at.

I've settled on spending time finding what works for me. Earlier today I took 30 minutes to add functionality to an app that would've taken me days to write. And what's more I only put 30 minutes into the diary for it, because I knew what I wanted and didn't care how it got there.

This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed.

rlewkov 2 days ago

> "This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed." So, if one's goal is to develop code that is easily maintainable by others, do you think that AI writing code gets in the way of that goal?

searls 2 days ago

Yeah, and then you just wind up feeling exhausted AND unsatisfied with where you wind up. You are exactly who I posted this for.

100% of my positive experiences with agent coding are when I don't have reason to care about the intrinsic qualities of the code (one-off scripts or genuine leaf node functions that can't impact anything else.

bakkoting 2 days ago

Anthropic's guide to using Claude Code [1] is worth reading.

Specifically, their recommended workflow is "first ask it to read the code, then ask it to make a plan to implement your change, then tell it to execute". That sounds like the workflow you're asking for - you can read its plan and make adjustments before it writes a single line of code.

One of the weird things about using agents is that if they're doing things in a way you don't like, including things like writing code without first running the design by you, you can simply ask them to do things a different way.

[1] https://www.anthropic.com/engineering/claude-code-best-pract...

woah 2 days ago

> you can simply ask them to do things a different way

Instead of a writing a blog post about how they didn't guess how you wanted things done?

ta12653421 2 days ago

good one!

I#m wondering how some can complain about ClaudeAI: - its actually enlightening - it saves a lot of time - by intuition, i did whats written in this blog from the beginning on

YES: - sometimes the solution is rubish because i can see that its "randomly" is connecting/integrating stuff - ...but: In about 95% of the cases the output is exactly what i asked for

bgro 2 days ago

Your ai is generating 2000 line code chunks? Are you prompting it to create the entire Skyrim game for SNES? Then after taking long lunch, getting mad when you press run and you find out it made fallout with only melee weapons in a ps1 style?

tristor 2 days ago

> It's like working with someone on their first day.

This matches my experience exactly, but worse than working with a human on their first day, day 100 for an AI is still like working with them on their first day. Humans have effectively infinite context windows over a long enough period of time, AIs context windows are so constrained that it's not worthwhile to invest the effort to 'teach' it like you would a junior engineer.

SirHumphrey 2 days ago

It’s not really that humans have infinite context windows, it’s more that the context windows are a very poor substitutes for long term memory.

Memory even in a text heavy field like programming is not only text based so it’s often hard to describe for example an appropriate amount of error checking in prompt.md. Giving a person with anterograde amnesia a book of everything they know - no matter how well indexed or how searchable will not fix the lack of long term memory.

theshrike79 2 days ago

The trick is to have rules specific to the project and your programming style & preferences.

Think of the AI like an outsourced consultant who will never say no. It'll always do everything it can to solve the problem you've given it. If it doesn't know how, it'll write a thousand lines when a single additional dependency and 2 lines of code would've done it.

slfnflctd 2 days ago

> "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here"

This is the gist of what I've always wanted from a programming mentor, instructor, or tutor.

It can be surprisingly hard to find. Knowing that current LLMs still struggle with it perhaps helps explain why.

chermi 2 days ago

I prompt it to come up with 3 potential implementation plans, choose which one it thinks is best, and explain its plan to me before it does anything. I also ask it to enumerate which files/functions it will modify in its chosen plan. Then you can give feedback on what it thinks and have it come up with a new plan if you don't like it. Every bit of context and constraints you give it helps. Having a design doc + a little description of your design/code "philosophy" helps. This is easy to do in cursor with rules, I'm guessing other tools have a similar feature. Also, if there's a somewhat similar feature implemented already or if you have a particular style, tell it to reference example files/code snippets.

flashgordon 1 day ago

This is kind of why I hardly use one shot/Agentic flows that the cursors are recomending you do (after all it gets them more control and lockin). I mostly know "how" I want something done so put fences to ensure what is generated is in the X loc ballpark and in an incremental way. They are however great to reason through ideas and explore solutions before writing any code.

zild3d 2 days ago

> I want an AI that I have similar opinions to, which is obviously tough. It's like working with someone on their first day.

Most of what you're describing does apply to humans on the first day, and ais on their first day. If you aren't capturing these preferences somewhere and giving it to either a human or the ai, then why would they somehow know your preferences? For ai, the standard thats forming is you create some markdown file(s) with these so they only need to be explained once, and auto provided as context.

Spooky23 2 days ago

Personally, I’ve gone from working with the AI to code to working on it to develop specifications. It’s also useful at troubleshooting issues.

I’m no longer a developer by trade and it’s more a hobby or specific problem solving scenario now. But I find using it to identify gaps in my thinking and edit English is ultimately better than getting random code — I love editing English text, but find editing code without consistent style a drag for my purposes.

ozim 2 days ago

At the start of prompt before project requirements I copy paste paragraph about the code I want.

No emojis, no comments, no console log statements, no read me file, no error handling. Act as a senior developer working with other experienced developers.

Otherwise it happily generates bunch of trash that is unreadable. Error handling generated will most of the times just hide errors instead of actually dealing with them.

ta12653421 2 days ago

initial prompting like this has a huge impact, yes.

also: I clean the chat and start over sometimes, because results may differ.

schwartzworld 2 days ago

> but because it's gone off and written a 2,000 lines of code

That’s a you problem, not an AI problem. You have to give it small tasks broken down the same way you would break them down.

nixpulvis 2 days ago

I honestly don't expect to use AI tools extensively for code generation until we figure out how to have the models learn and become accustomed to me aside from clever context prompting. I want my own models derived from the baseline.

That said, I also value not becoming too dependent on any service which isn't free and efficient. Relying on a CNC machine when you never learned how to whittle strips me of a sense of security and power I'm not comfortable with.

cmrdporcupine 2 days ago

I don't have this experience with Claude, frankly. I do have to correct it, but I try to give very specific prompts with very specific instructions. IT does well with highly commented code.

Now, I have the best luck in my personal project codebase, which I know extremely intimately so can be very surgical with.

Work, which has far less comments and is full of very high level abstractions that I don't know as well.. it struggles with. We both do.

It's a fine pair programmer when one of the pairs knows the codebase extremely well. It's a bad companion elsewhere.

Horde 2 days ago

There is an old issue, that's related, why one should avoid using equality relations in AI used for creating a proof. It will go back and forth and fill the log with trivial statements before it comes to the right proof path. This might end up being the majority of the proof and could be an unbounded part. Then somebody has to read that and spend a good deal of time deciding what's trivial and what isn't. Whereas if you remove equality, you have something that isn't very natural.

artursapek 2 days ago

You need to tune it.

jstummbillig 2 days ago

People are thinking too much of humans and LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort). This is simply not the case when doing AI coding, and you need to adjust how you work because of that and play to the strengths of this setup, if you want to get something out of it and not frustrate yourself.

Here is how to do this: Have it generate something. That first 2000 lines of not so great first attempt code, don't even think about understanding all of that, or, worse, about correcting it.

Review it loosely. You are not dealing with a human! There is absolutely no need to be thorough or nice. You are not hurting any feelings. Go for 80/20 (or the best ratio you think you can get).

Then, think:

- Anything you missed to inform the AI about? Update your initial prompt

- Anything the AI simply does not do well or to your liking? Write general instructions (all of the IDEs have some way of doing that) that are very explicit about what you don't want to see again, and what you want to see instead.

Then revert everything the ai did, and have it go again from the start. You should approach something that's better.

Sharlin 2 days ago

This approach is essentially the PR workflow preferred by the author. Why let an LLM make huge changes to your working copy just for you to revert them next, instead of just writing patches to be asynchronously reviewed? What you propose is no way of doing pair programming in particular, and seems to support the author’s argument.

jstummbillig 2 days ago

1. There is not a mention of "pair programming" in the comment I was addressing. As often happens, the discussion evolves.

2. The point is, that you are training the AI through this process. You can do pair programming afterwards (or not). Aim to instruct it to give you ballpark answers first, and take it from there.

afavour 2 days ago

> LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort)

At this point AI generated code absolutely requires review by a human so LOC is still an important metric.