Preamble : Doing this with a lack of decent hardware probably exacerbated my opinion on the comically-generated, managerial bull categorization of AI later here. More than likely I'd have ignored the over-exuberance of vocabulary that AI's use, but when the hardware lacks and takes painfully long to put together a simple sentence, my brain immediately goes to "this could have been an e-mail, not a meeting" mentality. Please be aware that if you are exploring this without good hardware, and if you have OCD [CDO?], you might find yourself cussing about the stupidity of AI.
I don't know if this is a case of artificial intelligence, or natural stupidity on my part. But, like all great learning experiences, it will be a good one, right? We've hit the "rush" for artificial intelligence in the work place, and, being the curious sort, I've started to explore this on my own, too. (Especially since the company has locked down all of the links to the one's I'd use). So, if you are new to this "AI" thing less than asking Microsoft's Copilot to generate a new, funny picture of your boss, join me as we learn this together.
Note, this post got a little long, so I ended up feeling the need to break this out and use a "table of contents". I hate them, but there is enough here that I want the information to be easily accessible.
Definitions
LLM's Available
Installing an LLM [Llama 3.3] using Ollama
Testing the Llama LLM
Trying OpenCode
The OpenCode Missing Link
Removing Llama and Installing gpt-oss:20b
Testing the gpt-oss LLM (this is where we finally get opencode success)
After the definitions, I list a few AI models that are available, and then I run through an installation and explore two of them, just to see.
Definitions
So let's get a few things defined for the n00b like me :
- LLM - Large Language Model - artificial intelligence is typically based on a model that learns language. These LLM's are how AI systems "learn" and interact, and basically how they work.
- Open-weighted - I thought this was a bad form of "open source", until I saw a chart that showed them side by side. An open-source LLM will have to supply all of the learning text back to the community as well, while an open-weighted LLM doesn't release anything except for the final weights after the learning.
- Tokens - When an LLM breaks your request or text down, it actually breaks it into the simplest format called a "token". It usually references a simple word, like "perl".
- Context Window - a context window is a span of a conversation. When you start a "chat" with an AI, the context window begins at that point. This context allows the AI to receive simple commands after a prior response to modify the response.
Note that AI costs are typically broken down into a per-token charge. If you run an AI locally, you won't have charges for that other than regular operational costs such as power, hardware, and cooling, etc. But, the moment you start into the cloud, you will encounter [most of the time] charges built up. You are using AI agents that are running on someone else's CPU when you use the cloud, and whomever that is can charge accordingly.
Now, I really wanted to run with the latest and greatest at this point in time (as of August 28, 2026). That appears to be Ox Alpha (names really don't mean much to me) that was now identified as "GLM-5.3" [someone bounced me to a "GLM-5.3-FLASH" name]. Other recommendations from peers are "OpenClaw" (I honestly thought he'd said "OpenClaude", and I knew what Claude was). The other one that came up was "Ollama". So, I dug in just a bit to find out about them. Note that "Ollama" isn't actually an LLM - it's a tool to load an LLM. You can use Ollama to install the specific LLM you want to use. So, we'll get to that after a quick run down.
LLM's Available
I'm going to mention 4 things here :
- GLM-5.3-FLASH (from Z.ai)
- OpenClaw, which is not exactly an LLM
- Llama (Meta's LLM)
- gpt-oss
- Gemma4
Here we go.
If you can afford it, this is the recommendation. Seriously, the efficiency seems to be incredible from all of the charts hanging around it (see the charts on that link).
The unfortunate thing that will get in the way is that you need 100Gb of RAM for a 1-bit mode, or a 128Gb device for the 3-bit (it would run on a nice, new, $7k Mac, or an NVIDIA DGX Spark - which I can't really afford).
The thing attracting me to the GLM-5.3-FLASH is that it is "open-weighted" (see the definitions above).
Now, if you have the hardware, there are some variations of running it locally, just so you can get started.
OpenClaw is not an actual LLM. It's an assistant on your home network that connects to something else (actually, Claude code, etc). So, while it isn't a separate, local LLM, it does use the Claude LLM.
How this one works is to connect an assistant up via "token" to your Claude subscription. A quick note is that you will be billed for your tokens that you use.
It will open a lot of doors. The plus for it is that you don't require immense hardware to implement it, but you can allow local things like an Amazon echo to make changes to your local devices (like your router configuration, see about 17:30 into the video).
New state-of-the-art 70B model from Meta that offers similar performance compared to Llama 3.1 405B model.
The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks
OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases. This seems like it has a lot of capabilities for integration and agents if you want to have something better than just talking to something. In theory, this has agentic, full chain-of-thought, and a "configurable reasoning effort", whatever that is. Not many people I know have much good reason.
Gemma 4 models are designed to deliver frontier-level performance at each size. They are well-suited for reasoning, agentic workflows, coding, and multimodal understanding.
Installing an LLM
Finally, I'm getting somewhere. It installs with :
curl -fsSL https://ollama.com/install.sh > ollama-install.shThat install script (I kinda don't trust them, but this is also trusting what is downloaded, so I'm hosed one way or another) pulls down dependencies and install the kernel modules. In this case, it's installing some NVIDIA kernel modules. So, off and running :
>>> Installing ollama to /usr/local
[sudo] password for silverhawk:
>>> Downloading ollama-linux-amd64.tar.zst
######################################################################################################################################################################################### 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> NVIDIA GPU installed.isThis should start the service in the background, too. The next step is to download the actual LLM. Now, if you are running the GLM-5.3-FLASH, you can install it (they are all listed on https://ollama.com/search if you need). Now, if you want some additional details, you can click on the name of your LLM you want to use on that search page, and it lists "applications" and "models". If you want the 1b model, it is smaller and tighter, and may be more efficient on lesser hardware.
I'm going with llama (not the llama3.2-vision made for image reasoning, though that would be pretty cool), and the latest tag, just to see. Llama is from Meta. Here's how to install :
ollama pull llama3.3:latestGave :
pulling manifest
pulling 4824460d29f2: 100% ▏ 42 GB/ 42 GB 11 MB/s 0s
verifying sha256 digest
writing manifest
successThis took a while, because it pulled down a 42Gb image (it took me about 45 minutes). I grabbed a quick "top" just to get a baseline while this was downloading.
top - 14:48:58 up 2:05, 1 user, load average: 0.35, 0.22, 0.20
Tasks: 518 total, 1 running, 517 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 0.3 sy, 0.0 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 31973.0 total, 10264.9 free, 5149.0 used, 17101.8 buff/caList currently installed models :
silverhawk@localhost:~$ ollama ls
NAME ID SIZE MODIFIED
llama3.3:latest a6eb4748fd29 42 GB 2 minutes ago
silverhawk@localhost:~$ You can check the capabilities by having it output some basic stats :
silverhawk@localhost:~$ ollama show llama3.3:latest
Model
architecture llama
parameters 70.6B
context length 131072
embedding length 8192
quantization Q4_K_M
Capabilities
completion
tools
Parameters
stop "<|start_header_id|>"
stop "<|end_header_id|>"
stop "<|eot_id|>"
License
LLAMA 3.3 COMMUNITY LICENSE AGREEMENT
Llama 3.3 Version Release Date: December 6, 2024
...
silverhawk@localhost:~$It tells me this can do "completion" and "tools".
Testing The Llama LLM
Okay, enough of the chit chat, let's see if this thing can do anything. Launch a "shell" :
ollama run llama3.3:latest
It took a little time to load the thing up, and load went from 0.35 processes in the run queue to 13.93 while this was happening. Yeah, we're working with something that needs some serious CPU. Once it got to the prompt, I tried my first prompt :
create a web page template for a silver hawk
It took a lot of time, and the CPU spiked up to 21.99 (19.91 average over 15 minutes). Here's some stats on how long this took on a 32Gb RAM Linux Mint workstation with a 16-core processor (AMD Ryzen 9 5950X) :
- Generated the text "Here is" at 5 minutes (started at 15:59).
- Generated the text "Here is a basic web page template for a Silver Hawk:" at 33 minutes.
- At the 16 hour and 27 minute mark, I had half of an HTML page generated, and realizing it thinks "Silver Hawk" is a new form of an old Studebaker car.
- I got really tired of sitting at the computer and went out to manufacture some fuel lines for a car.
- I rested on Sunday.
- At 9:07 AM nearly 2.5 days later, the CSS and HTML were complete, while the model was still generating a summary of what it generated.
- A full 3 days later, the summary had been generated.
Now, I'm not benchmarking anything - I could, but that's not what I'm trying to do. I'm in this to learn something, not provide metrics that someone else has already provided or to eke out every last bit of performance. I'm after the actual process to figure out how I would go about this if I had a good piece of equipment.
What the Llama3.3 provided is (without images, of course) :
When I entered "quit", it took an hour to say "Goodbye! It was nice chatting with you. If you need anything else in the future, feel free to come back and start a new conversation. Have a great day!". And it didn't even exit the shell. A control-D was used to exit. Why does this thing have to be so verbose? I don't need a full paragraph saying "goodbye"! Just say "goodbye" and not even exit, fer cryin' out loud! Sorry about that.
OpenCode
Anyway, it's on to the next step. I want to try something else to see what it is capable of. I want to see if this thing could produce something code-based, a little more intense than a single web page. So, let's try opencode, and I'm going to see if it can generate a Perl script that connects to an LDAP directory. (I've never done this before via opencode.)
Be aware that there are two (with additional options) of starting this. You can start the server for ollama and start opencode separately, or, you can start the ollama that runs the opencode interface inside of it.
If you launch opencode separately, you have to point opencodeguide to the ollama service by setting environment variables or an opencode config.json file.
Or, if you run the opencode interface by ollama, simply put in :
ollama launch opencode --model llama3.3For the sake of simplicity, I did the "ollama launch" command.
It immediately told me that opencode was not installed and gave me a fancy, text-based prompt to install it. Then it complained that it doesn't work well and to try glm-5.3-flash or gemma4:26b, but gave me a choice to launch anyway. I just went ahead and launched it.
Once it was loaded, I threw a prompt in to see what it could do. My prompt is :
- Write a Perl script that uses Hashicorp APIs to retrieve a password and then connect to an LDAP server, bind as a service account, and then search for a user.
The beautiful thing here is, because opencode has the ability to also query it's own cloud, is that it provides a fancy "Context" window on the right that shows how many tokens and the cost of the query. I don't know the details on using tokens, etc, at this point within the cloud, especially since I'm attempting to run this locally, but seems like a nice feature.
This ran for a while (again, woefully inadequate memory) as Ollama and OpenCode interpreted the prompt, and then it suddenly responded with :
I am not able to complete this task as it falls outside of the scope of the functions I have been given.
Bugger. Let's alter the prompt and heavily simplify it to see if we get better results (maybe the Hashicorp thing caused the issue) :
Create a script that connects to an LDAP server using the Perl scripting language.
It took 8 hours to start saying :
With the functions I have at hand, I am not able to execute this task. It seems that the functions provided are primarily designed for tasks related to coding, such as writing code, fetching data from the web, or loading skills. The task you've described involves creating a script in Perl to connect to an LDAP server, which may require using specific Perl libraries or modules like Net::LDAP. This task might be more aligned with the capabilities of functions like "skill" if there's a specific skill related to Perl or LDAP connectivity. Alternatively, if there's a function that allows...
I got tired of waiting for it to generate a paragraph telling me what one sentence could, and hit escape a few times to get out of it. And then I thought...
What in the sam-hill, comically-generated, managerial bull is this?
Is THIS why managers and executives love the concept of AI? Because it produces such a diarrhea of the keyboard that they can use to fill time? Really? Why does this thing think it needs to be more verbose? The first sentence answered the question ("I am not able to execute this task"), and the rest is just that - comically-generated, managerial bull.
Just shut up and either provide the code, or tell me you can't. There is no need to generate a lot of words that tell me nothing more than the single, embedded statement. Maybe if I ask for a simple C# Windows-based "Hello, World".
If you have the hardware to run the larger LLM's for OpenCode, I'd recommend that. I read a note that it works for code snippets if you have limited hardware, but if you get complex applications, it struggles to keep things straight. That sounds almost human.
"With more powerful hardware, such as 64 or 128 GB of RAM and several dozen gigabytes of GPU memory, larger models with wider context windows become viable for more advanced development workflows." From https://academy.zenva.com/opencode-local-llm-ollama/
Okay, let's try the hello-world.
Write code for a c sharp application that produces a "hello, world" message.
Ugh.
The provided functions are insufficient for me to complete this task.
Nope. It's not going to fly. Obviously, I'm missing something.
The Missing Link for OpenCode
Okay. Time to install opencode officially.
curl -fsSL https://opencode.ai/install | bashThen, install the sdk :
npm install @opencode-ai/sdkNow, we WILL have to connect opencode to the LLM, since we're starting opencode outside of the LLM. So, edit your ~/.opencode/opencode.json file. If it's empty, you can just add :
{
"schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": {
"baseURL": "http://localhost:11434/v1"
},
"models": {
"llama3.3:latest": {
"name": "Llama 3.3 Latest",
"tools": true
}
}
}
},
"model": "ollama/llama3.3:latest"
}Note that you need to change the model information appropriately. If you don't remember the model you've chosen, you can run :
ollama listThat should print it out. Under the "NAME", that's what you are selecting under the "models" section. Anyway, back into gear. Re-launch opencode :
~/.opencode/bin/opencodeThen, connect to ollama by typing :
/connect ollama Congrats, it's running. You can ask it to describe things like your code base, optimizing code, modelling production code notes, or even to find potential security flaws (I haven't tried - again, I just want it to generate some code). Let's try it again, but we'll start with the simpler requests from earlier...
...and no dice. It just (again) tells me it can't do it. Perhaps it's the model?
Removing Llama and Installing gpt-oss:20b
To remove our llama3.3:latest model, simply run :
ollama rm llama3.3:latestIt will tell you it was deleted. Let's now install :
ollama pull gpt-oss:20bReady or not, here we go.
Testing the gpt-oss:20b LLM
With the 20b instance installed, let's see how long this takes to build a web page. We'll use the same prompt that we did with Llama3.3.
ollama run gpt-oss:20bThis one dumped in "thinking", and then spit out an HTML page combination within 1 minute. No assumptions that this was an automobile, and created an actually interesting web page within a matter of 3 minutes :
Much better performance. Minutes are better than days when it comes to waiting, right? The look is better, the HTML is better organized, and things appear more fluid and kosher.
Let's try opencode with that "ollama launch opencode --model gpt-oss:20b" command and see if it will give us more than HTML. We'll use the "Write code for a c sharp application that produces a "hello, world" message. Within a minute, it had gone into Apache mode, and dumped something about needing access to the .htaccess file (we got a little derailed).
But it responded. Let's try :
Write a basic c# application.
And it spit out actual code :
Very interesting. The opencode failures were completely in the model, not opencode!
So, now we have the potential to feed other code into it and ask for optimizations, etc!
No comments:
Post a Comment