A full (slow) solution to the Billion Row Challenge https://video.infosec.exchange/videos/watch/863d5155-0530-4783-8c62-84ba2b15be04 We've been picking off small sub-tasks of the Billion Row Challenge, which asks us to summarise a large amount of data as quickly as we can. This time we complete the whole challenge, but in a slow way, to make sure we understand it. In future videos we will attack the last remaining sub-tasks, which concern how to collect together the information (probably in a hashmap) and print a summary. The code is at https://codeberg.org/andybalaam/brrmbrrm Read my blog at https://artificialworlds.net/blog Follow me on mastodon: @andybalaam@mastodon.social Sat, 11 Jul 2026 02:29:04 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://video.infosec.exchange A full (slow) solution to the Billion Row Challenge https://video.infosec.exchange/client/assets/images/icons/icon-1500x1500.png https://video.infosec.exchange/videos/watch/863d5155-0530-4783-8c62-84ba2b15be04 All rights reserved, unless otherwise specified in the terms specified at https://video.infosec.exchange/about and potential licenses granted by each content's rightholder. <![CDATA[A full (slow) solution to the Billion Row Challenge - Andy Balaam]]> https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 Fri, 10 Jul 2026 08:22:10 GMT @janWilejan@snug.moe (stick to what I did for count, specifically)

]]> Andy Balaam <![CDATA[A full (slow) solution to the Billion Row Challenge - Andy Balaam]]> https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 Fri, 10 Jul 2026 08:21:44 GMT @janWilejan@snug.moe Yes, good point - we could use i16 for min and max. I'd prefer not to assume anything about the distribution of stations so will stick to what I did (i64?).

A perfect hash function is cheating. We have a test input file but we're trying to write code that would work with any similar file.

There's a video of me crafting my .vimrc here: https://video.infosec.exchange/w/rUhu39ciya1F99ZziK5Gun and the code is here: https://gitlab.com/andybalaam/configs/-/tree/main/.config/nvim?ref_type=heads

]]> Andy Balaam <![CDATA[A full (slow) solution to the Billion Row Challenge - jan Wilejan]]> https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 https://video.infosec.exchange/w/hzreVwHv7hETKUG82KtGGd;threadId=270655 Fri, 10 Jul 2026 06:30:22 GMT @andybalaam some thoughts: i think you can use i16 for the min/max and i32 for the count. (assuming no city is more than 0.2% of the billion rows and the average temp is less than 99.9 (1e9*.002*999 < 2**31)).

also is using a perfect hash function cheating? if you know the city names in advance, you can avoid performance hits due to hash table collisions.

also, is your vim config posted anywhere?

]]> jan Wilejan