<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0"
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:atom="http://www.w3.org/2005/Atom">
    <channel>
        <title>awk — LowEndSpirit</title>
        <link>https://lowendspirit.com/</link>
        <pubDate>Sun, 23 Aug 2026 17:23:17 +0000</pubDate>
        <language>en</language>
            <description>awk — LowEndSpirit</description>
    <atom:link href="https://lowendspirit.com/discussions/tagged/awk/feed.rss" rel="self" type="application/rss+xml"/>
    <item>
        <title>Parsing YABS output with awk</title>
        <link>https://lowendspirit.com/discussion/11253/parsing-yabs-output-with-awk</link>
        <pubDate>Fri, 21 Aug 2026 09:00:00 +0000</pubDate>
        <category>Blog</category>
        <dc:creator>mikho</dc:creator>
        <guid isPermaLink="false">11253@/discussions</guid>
        <description><![CDATA[<p>You know the thread. Someone's shopping for a cheap VPS, they ask "which is faster, Contabo or RackNerd or the new Hetzner box," and six people paste their full <code>yabs.sh</code> output in reply. Each one is forty-plus lines of system info, fio disk tables, iperf3 network results, and a Geekbench 6 score buried near the bottom. Scrolling through five of these to compare single-core numbers is how you lose twenty minutes you didn't mean to spend.</p>

<p>Here's what actually gets pasted, trimmed to the parts that matter:</p>

<pre><code>### Contabo VPS S SSD (Nuremberg)

fio Disk Speed Tests (Mixed R/W 50/50) (Partition /):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 61.14 MB/s   (15.2k) | 214.30 MB/s   (3.3k)
Write      | 61.35 MB/s   (15.3k) | 216.02 MB/s   (3.3k)
...

Geekbench 6 Benchmark Test:
---------------------------------
Test            | Value
                |
Single Core     | 812
Multi Core      | 2431
Full Test       | https://browser.geekbench.com/v6/cpu/7710214
</code></pre>

<p>That <code>### Contabo VPS S SSD (Nuremberg)</code> header isn't from YABS, it's whatever the poster typed above their paste to say which box this is. Everything below it is the script's own formatting, and it's consistent enough between runs that awk can chew through it without much fuss.</p>

<h3>The quick version: just the Geekbench numbers</h3>

<p>If all you want is single-core and multi-core scores lined up so you can eyeball which box is actually faster:</p>

<pre><code>awk -F'|' '
/^### /{p=$0; sub(/^### /,"",p)}
$1~/^Single Core[ \t]*$/{s=$2; gsub(/^[ \t]+|[ \t]+$/,"",s)}
$1~/^Multi Core[ \t]*$/{m=$2; gsub(/^[ \t]+|[ \t]+$/,"",m); printf "%-30s single=%-6s multi=%s\n", p, s, m}
' pasted-results.txt
</code></pre>

<p>Run against three pasted YABS blocks, that produces:</p>

<pre><code>Contabo VPS S SSD (Nuremberg)  single=812    multi=2431
RackNerd VPS (Los Angeles)     single=1189   multi=1972
Hetzner CX22 (Falkenstein)     single=1204   multi=2298
</code></pre>

<p>The trick is the <code>-F'|'</code> field separator. YABS lays its tables out with pipes, so once you split on those, <code>Single Core     | 812</code> becomes two fields: <code>$1</code> is the label with trailing spaces, <code>$2</code> is the value with a leading space. The <code>gsub</code> calls strip that whitespace so you're left with a bare number. The header pattern (<code>/^### /</code>) resets which provider name gets attached to the next score it finds, so as long as your pasted results keep that header convention, each row in the output lines up with the right box.</p>

<h3>Adding disk speed to the comparison</h3>

<p>Geekbench alone doesn't tell you if the disk is going to choke under real load, and on a lot of $3-a-month VPS plans, the disk is the actual bottleneck, not the CPU. The fio section has the same pipe-delimited shape, just with more noise around it: the 4k block numbers come with IOPS counts in parentheses that you don't want mixed into your speed comparison.</p>

<pre><code>BEGIN {
    FS = "|"
    printf "%-28s %8s %8s %14s %14s\n", "Provider", "Single", "Multi", "4k Read", "4k Write"
}
/^### / {
    provider = $0
    sub(/^### /, "", provider)
    in4k = 0
}
/Block Size[ \t]*\| *4k/ { in4k = 1 }
/Block Size[ \t]*\| *512k/ { in4k = 0 }
in4k &amp;&amp; $1 ~ /^Read[ \t]*$/ {
    val = $2
    match(val, /[0-9.]+ *[A-Za-z\/]+/)
    read4k = substr(val, RSTART, RLENGTH)
}
in4k &amp;&amp; $1 ~ /^Write[ \t]*$/ {
    val = $2
    match(val, /[0-9.]+ *[A-Za-z\/]+/)
    write4k = substr(val, RSTART, RLENGTH)
}
$1 ~ /^Single Core[ \t]*$/ {
    single = $2
    gsub(/^[ \t]+|[ \t]+$/, "", single)
}
$1 ~ /^Multi Core[ \t]*$/ {
    multi = $2
    gsub(/^[ \t]+|[ \t]+$/, "", multi)
    printf "%-28s %8s %8s %14s %14s\n", provider, single, multi, read4k, write4k
}
</code></pre>

<p>Save that as <code>parse-yabs.awk</code> and run <code>awk -f parse-yabs.awk pasted-results.txt</code>. Against the same three-provider paste:</p>

<pre><code>Provider                       Single    Multi        4k Read       4k Write
Contabo VPS S SSD (Nuremberg)      812     2431     61.14 MB/s     61.35 MB/s
RackNerd VPS (Los Angeles)       1189     1972    143.67 MB/s    144.13 MB/s
Hetzner CX22 (Falkenstein)       1204     2298    268.90 MB/s    270.02 MB/s
</code></pre>

<p>Now the whole comparison is one glance instead of three separate walls of text. Contabo's CPU numbers look fine on paper, but that 61 MB/s on 4k random writes is the kind of thing that explains why a database on that box feels sluggish even when the benchmark score says it shouldn't.</p>

<p><img src="https://lowendspirit.com/uploads/editor/m6/fou5v9fogvmd.webp" alt="LowEndSpirit - VPS Hosting and tech forum" title="" /></p>

<p>The <code>in4k</code> flag is doing the real work here. YABS runs the fio test at four block sizes (4k, 64k, 512k, 1m) in the same table, and the Read/Write labels repeat for each one. Without tracking which block-size section you're currently inside, the script would just grab whichever Read/Write pair it saw last, which on a four-block table is the 1m numbers, not the 4k ones you probably care about for a general-purpose VPS. Flip the two <code>Block Size</code> patterns near the top if you'd rather track 512k or 1m instead.</p>

<h3>Where this breaks</h3>

<p>YABS output format has changed between versions before, and it'll probably change again. If a future release of <code>yabs.sh</code> reformats the fio table or renames "Single Core" to something else, this script silently produces blank columns instead of erroring, which is worse than a crash because you might not notice. Check the output against the raw paste the first time you run it against a new YABS version, after that it's fire and forget for as many providers as people keep dropping into the thread.</p>

<p>The other limitation: this assumes everyone pastes their result under a <code>### provider name</code> header. Not everyone does. If someone just pastes raw <code>yabs.sh</code> output with no label, the script will still grab their numbers, just under whatever the previous header was, which is wrong. Worth a note at the top of your comparison thread asking people to label their pastes, it saves you from a script that fails quietly.</p>
]]>
        </description>
    </item>
    <item>
        <title>sed vs awk vs grep: when to reach for which</title>
        <link>https://lowendspirit.com/discussion/11251/sed-vs-awk-vs-grep-when-to-reach-for-which</link>
        <pubDate>Sat, 22 Aug 2026 07:00:00 +0000</pubDate>
        <category>Blog</category>
        <dc:creator>mikho</dc:creator>
        <guid isPermaLink="false">11251@/discussions</guid>
        <description><![CDATA[<p>Every terminal tutorial teaches these three with the same tired example: <code>cat file | grep foo</code>. <br />
That's not a real task, it's a syntax demo.</p>

<p>On an actual VPS, the question isn't "how does grep work," it's "which of these three do I even use?" Here's how to answer that in about three seconds, using a real <code>access.log</code>.</p>

<p>Say your box just had a slow morning and you're digging through last night's nginx access log:</p>

<pre><code>203.0.113.14 - - [18/Aug/2026:03:12:01 +0000] "GET /wp-login.php HTTP/1.1" 404 162 0.002
198.51.100.7 - - [18/Aug/2026:03:14:22 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.014
198.51.100.7 - - [18/Aug/2026:03:14:23 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.011
203.0.113.14 - - [18/Aug/2026:03:15:40 +0000] "POST /xmlrpc.php HTTP/1.1" 404 162 0.002
192.0.2.55 - - [18/Aug/2026:03:20:10 +0000] "GET /api/v1/status HTTP/1.1" 200 48 3.821
192.0.2.55 - - [18/Aug/2026:03:20:15 +0000] "GET /api/v1/report?range=30d HTTP/1.1" 200 118402 1.204
198.51.100.7 - - [18/Aug/2026:03:22:01 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.009
203.0.113.14 - - [18/Aug/2026:03:23:55 +0000] "GET /.env HTTP/1.1" 404 162 0.001
192.0.2.55 - - [18/Aug/2026:03:25:30 +0000] "GET /api/v1/report?range=7d HTTP/1.1" 200 41200 0.412
</code></pre>

<p>The last column is response time in seconds, added by a custom log format. That's the line that pays off later.</p>

<h3>Question one: am I looking for lines, or looking at data inside them?</h3>

<p>If the answer is "I just need to find the lines," that's grep. Nothing else. You want every request that hit your status endpoint:</p>

<pre><code>$ grep '/api/v1/status' access.log

198.51.100.7 - - [18/Aug/2026:03:14:22 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.014
198.51.100.7 - - [18/Aug/2026:03:14:23 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.011
192.0.2.55 - - [18/Aug/2026:03:20:10 +0000] "GET /api/v1/status HTTP/1.1" 200 48 3.821
198.51.100.7 - - [18/Aug/2026:03:22:01 +0000] "GET /api/v1/status HTTP/1.1" 200 48 0.009
</code></pre>

<p>Four lines out of ten, full text, unmodified. That's the whole job of grep: a filter, not a processor. The moment you catch yourself piping grep's output into <code>cut</code> or <code>awk '{print $1}'</code> to pull a field back out, stop, you skipped a step. You didn't need lines, you needed data.</p>

<p><img src="https://lowendspirit.com/uploads/editor/xf/arvs39622270.webp" alt="LowEndSpirit - VPS Hosting and tech forum" title="" /></p>

<h3>Question two: do I need to compute something across fields?</h3>

<p>That 3.821-second response on line five is the actual problem, one request from 192.0.2.55 that took four thousand times longer than the others. Finding that by eye in a real log with fifty thousand lines isn't happening. This is awk's job, because awk thinks in fields and running totals, not just matched lines:</p>

<pre><code>$ awk '{ bytes[$1]+=$10; if ($NF+0 &gt; slow[$1]) slow[$1]=$NF }
       END { for (ip in bytes) printf "%-15s bytes=%-8d slowest=%.3fs\n", ip, bytes[ip], slow[ip] }' access.log

198.51.100.7    bytes=144      slowest=0.014s
192.0.2.55      bytes=159650   slowest=3.821s
203.0.113.14    bytes=648      slowest=0.002s
</code></pre>

<p>One pass over the file, running two accumulators keyed by IP, and the answer falls out: 192.0.2.55 is both your heaviest bandwidth consumer and the source of that slow request. grep could never have told you that. <br />
It doesn't do arithmetic, and it doesn't remember anything between lines. awk does both by default. <br />
That's the actual dividing line between them: grep answers "which," awk answers "how much," "how many," or "what's the total."<br />
<img src="https://lowendspirit.com/uploads/editor/5k/vtizrd5cfq64.webp" alt="LowEndSpirit - VPS Hosting and tech forum" title="" /></p>

<h3>Question three: do I need to change the file itself?</h3>

<p>Neither grep nor awk touches your file; they both just read it and print to stdout. If the task is "the box just got renamed, and I need to update the nginx config to match," that's sed, because sed's whole reason to exist is in-place transformation:</p>

<pre><code>$ cat nginx-site.conf
server {
    listen 80;
    server_name oldbox.example.net www.oldbox.example.net;
    ...
    proxy_set_header Host oldbox.example.net;
}

$ sed -i.bak 's/oldbox\.example\.net/newbox.example.net/g' nginx-site.conf

$ cat nginx-site.conf
server {
    listen 80;
    server_name newbox.example.net www.newbox.example.net;
    ...
    proxy_set_header Host newbox.example.net;
}
</code></pre>

<p>Three occurrences, one command, and the <code>.bak</code> suffix means the original is still sitting right next to it if the regex was wrong. (It wasn't, but check anyway, every time, no exceptions.)<br />
<img src="https://lowendspirit.com/uploads/editor/te/ex4mxd37qpc0.webp" alt="LowEndSpirit - VPS Hosting and tech forum" title="" /></p>

<h3>The actual decision tree</h3>

<p>Not three bullet points with bold headers, just the real question in your head when you open a log file: <strong>do you need to find lines, compute across fields, or edit the file in place?</strong></p>

<ul>
<li>Find lines, grep.</li>
<li>Compute or reshape data, awk.</li>
<li>Change the file, sed.</li>
</ul>

<p>Most real admin tasks are one of these, cleanly, and the rare ones that feel like two at once (find matching lines <em>and</em> count them) are just awk wearing grep's job for a minute, since awk can filter with a pattern before its action block just fine.</p>

<p>You'll still combine grep and awk plenty, and that's fine. grep filters the lines first, so awk only has to work through what's left, which matters on a big file. The real skill isn't picking a favorite tool; it's noticing what you're actually asking for: a field, a total, a changed line. Once you can name that, the right tool is obvious. Most tutorials never make you ask a real question, so you never get the practice.</p>
]]>
        </description>
    </item>
   </channel>
</rss>
