The title is a bit hyperbolic but this is how I feel about it. Actually I am proud to say that, against any prediction, I was able to submit and get Speedata Publisher approved into the FreeBSD Ports tree.
Going back to lesser nerdy language…
My apologies, technology and nerdy stuff often go hand in hand, allow me to articulate a bit more:
Speedata Publisher is the main component of Advance Grid Layout, my template to build premium & professional hybrid (auto and semi-auto) large volume catalogs;
FreeBSD is the main operating system I use to run my business;
and the Ports tree is the FreeBSD repository (or archive) for system and third-party software
It shouldn't be secret but as a matter of fact, my workflow is entirely built on opensource software, however rather than being built upon GNU/Linux I elected the BSD world, and specifically FreeBSD.
FreeBSD is a Unix-like operating system, derived from the venerable Unix operating system; I elected it because it is reliable, robust, fun, and actively under development; FreeBSD has a lot of interesting features, the main one worth mentioning in this article is the total integration with the ZFS filesystem. The latter is the state of the art of copy-on-write (COW) filesystem, with plenty of features that is not possible to list in this article. What is extremely important is that ZFS guarantees by default:
Data integrity
Data snapshot
Data recovery
With FreeBSD and ZFS your data is protected and fully recoverable at any time and at any moment.
Why making a port?
Patrick from Speedata does the effort to maintain the Windows version, the Linux version (now only as tarball) and the MacOS version. That is frankly a lot. On my end I learned how to compile from source on GNU/Linux Debian for both AMD64 and ARM64 architecture and for FreeBSD AMD64, however since I work on different FreeBSD workstations, it was becoming tedious to keep Publisher updated everywhere, and therefore I started to feel the urge to make a port and have Publisher available directly from the packages repository.
This is where AI makes its entrance
Although FreeBSD folks state that porting a software is not complicated, after reading twice the Ports Handbook I realized that porting Publisher on FreeBSD wasn't easy at all, regardless the fact I was able to compile from source with some dirty tricks. Anyway this idea kept growing until I decided to test it out with the aid of some freely accessible AI/LLM models.
At the beginning it looked like these free models based on reduced versions of ChatGPT were good enough but I quickly hit two big obstacles:
They assume to know what they don't know at all.
They do not have enough memory to handle complex tasks.
I jumped from free service to free service, I used Duck.AI, ChatGPT, Gemini, and DeepSeek too. All of them looked promising until they started to hallucinate and losing context severely. While I was testing services, in the same period Claude was silently taking the leadership on the coding side, so I tested it out.
It immediately performed better, with better understanding and context but Sonnet wasn't good enough, so I moved to Opus. This was a true revelation: Opus was extremely better and superior in every aspect but once I started to groove deeper into the task, unfortunately, I quickly hit the limit of the free version! At this point I couldn't go back and I decided to switch to the premium account.
One step forward three steps back
AI tools, the good ones, are cool — if it wasn't for the terrific abuse of the environment though — and if you know what you are doing and you dominate the topic, they can really multiply what you can do. Actually, with Opus, I felt I could achieve the task, but I was right and I was wrong at the same time.
Without a prior experience of the ports mechanism, especially regarding GO, I had to rely completely on Opus. However, for everything we did good we did also three wrong steps, I was inside a extremely sophisticated loop, basically I fell again into the AI/LLM trap.
Misleading by design
What I am going to write I believe that is nothing new, as a matter of fact I would dare to say that is at least 20 years that any internet technology we use daily, whether are desktop or mobile, is built upon social engineering. However social engineering is not enough to explain what I meant — it is right from an engineer perspective but doesn't convey the point — the pervasive and invasive technologies that made the fortune of the biggest Tech Companies are built to exploit our human psychology. Plenty of psychologists and social behavior specialists work harder than ever to exploit our innate behaviors with the goal to trap people (aka end-users) into their apps and ecosystems.
The very same principles applies for the AI/LLM models too, these are built to exploit our trust, to take advantage of our naive trust in technology and the friendly attitude these models project, with the goal of lowering our critical thinking, sane skepticism, and eventually building trust.
This is extremely important because I fell for it too. Since I didn't dominate the topic, actually it was for me the first time addressing it, I found the tone of these models very convincing.
Systematically reporting errors to these models became progressively less effective as I got closer to narrowing down specific blocking points. When you reach these points the behavior of the models becomes inconsistent. They always find a newer solution that does not work, after several of this blocking situation, I started to realize better the AI/LLM behavior rather than the Ports tree process.
Not all the AI share the same level of efficiency
This can sound trivial but you can only measure it against the complexity of your task, so while any free service can produce an acceptable output, only paid service can go deeper into the task. And, not all the paid services have the same ability in specific areas, I tested some of the most known and mostly failed miserable. That told me they weren't trained enough on my peculiar area of interest; however, Claude did not fail at the first prompt, but Sonnet was immediately circling around without getting anywhere; although Opus did not.
The model makes the difference, actually what makes the real difference is its training sets, and apparently Anthropic made a good job with their coding training set, however also Opus — at that time the most advanced Anthropic model — wasn't good enough; and is here where I learned another interesting detail about AI/LLM.
AI/LLM aren't creatives and can't solve problems
AI/LLM models, the commercial ones, are built to give you the illusion or the feeling that they understand what they are doing, even when they repeatedly mistaking. During the phase of staging Publisher (one of the Ports procedure), I hit another impasse that appeared unsolvable: fed with the usual output, Opus continued to have, at every iteration, more original and better solutions that never worked out. Eventually I had to stop this loop, it was clearly hallucinating, unable to find a way out. Why?
I am not an AI/LLM expert, but the answer can be only one: if a model doesn't have a solution hidden somewhere in its datasets it is not capable by any means to abstract nor even to inference from a problem to another. So, how did I let Claude Opus solve it?
Since I was dealing with a GO project, I started to check for GO software in the Ports tree and I let Claude digest a couple of selected Makefiles, and in one of these files there was the solution. A solution that a human brain developed, based on real study, that was perfectly fitting my current block. This really gave me a clear understanding about how AI/LLM models work! These models work for patterns: if they have in their datasets a working pattern they can find the solution to your problem; if they do not have a working pattern, they simply approximate, or even worst hallucinate, without barely understanding the problem.
Submitting the port
Eventually with Opus aid I was able to build Publisher inside Poudriere without hiccups and I felt ready to submit it. The first attempt was catastrophic, I was badly scolded for incompetence and blamed for the use of AI, once I explained that I wasn't a Computer Scientist I received a more tolerant attitude and the willing to be helped in succeeding this task!
The first port submission had several trivial errors that I wouldn't have expected from Claude. The Ports mechanism is one of the finest, and Claude did not have any clue on how to prepare a good one, despite it claiming to have read the documentation several times.
However, based on my experience, the AI/LLM models really shine when fed with a working pattern, thus when the amazing guy assigned to evaluate my submission spotted all the errors, Claude was able to grasp better the big picture and was ready to apply the fixes; but eventually the guy assigned to my port decided to take the initiative and to close the port from his end.
Finally Speedata Publisher (stable) made its first appearance as an official package on FreeBSD, and its very first official entry in the BSD realm.
Power is nothing without control
Does this title sound familiar?
It matches extremely well with AI/LLM models, for what I learned during this process, without human intervention they can derail pretty soon and pretty bad as well. On my personal experience, the main point of using Claude was the fact that it works extremely well as interactive documentation, however it had a grasp of the Makefile but not enough to prepare an error-free one.
Nonetheless it learned very fast, and it was quick to replace a non-functional pattern with a better one, however the self evaluation is only an illusion. These tools miss a real discretionary judgment, you may have not noticing, but I did walk the complete parabola: from the wrong point to the right one. I witnessed all the fascinating journey of Claude, but what fascinated me the most is the importance of the human factor.
Without the guidance and experience of the Ports member, completing this task wouldn't have been possible. Maybe it would have taken 24 months to prepare another broken Makefile but with a substantial difference.
The trade off when you use AI/LLM models…
Although I am extremely happy about this personal achievement, I feel that I became good enough at squeezing Claude rather than at making ports. Actually, now after doing and redoing the process I've started to understand better the principles of structuring a Makefile but I am still way too far from even a barely modest knowledge of it.
Using Claude as interactive documentation and assistant helped to get the port quickly, and helped to make the process reproducible, but it really didn't help in terms of mastering the subject matter. Which is still fine from my point of view, but for everybody else, especially youngsters, that want to be more involved in the FreeBSD project I warmly recommend to avoid these tools and rely only on tried-and-true methods.
That said, I would not close the door to anybody who wants to contribute in some way, like adopting an orphan port, with the assistance of the right AI tools. Actually, a well trained model that can filter initial requests would alleviate a lot of work for the Ports staff, this is undeniable.
For the next future…
Speedata Publisher development is pretty conservative, I don't believe there will be any breakage from version to version, so for now I can feel comfortable the documentation will be fine for a while. I will continue to contribute to Publisher by making it available on FreeBSD, and indirectly I will help FreeBSD to be a compelling OS for automated desktop publishing.
If your are interested in learning more, please, do not hesitate to contact me and understand how AGL can help to improve the quality of your business and to produce your catalogs better and faster!
RenderData.pro
B2B product catalog automation powered by cutting-edge technology.
Contacts
Request a quote or an appointment.
+1 (786) 558-1444
info@renderdata.pro
© 2024. All rights reserved. Privacy.