Episode #612

Parallelize the work with Swift Concurrency

Series: Billion Row Challenge

48 minutes
Published on August 14, 2026

This video is only available to subscribers. Get access to this video and 611 others.

Get Started
Log In
We parallelize our billion-row parser using structured concurrency, splitting a memory-mapped file into newline-aligned chunks and processing them concurrently with a Swift Task Group before merging the per-chunk results. Along the way we cover @unchecked Sendable on pointer-backed types, why the SIMD experiment didn't pay off compared to memchr, benchmarking with hyperfine, and tuning chunk count against processor count. All of this ends up taking the run from 50 seconds down to about 5.

Links

Code

A Sendable wrapper around the memory-mapped file, so chunks can be handed to concurrent tasks:

struct MappedFile: @unchecked Sendable {
    let base: UnsafeRawPointer
    let size: Int

    var bytes: UnsafePointer<UInt8> {
        base.assumingMemoryBound(to: UInt8.self)
    }
}

Splitting the file into newline-aligned ranges, using memchr to walk forward from each naive split point to the next line feed:

func chunkRanges(in file: MappedFile, count: Int) -> [Range<Int>] {
    let targetSize = file.size / count
    var ranges: [Range<Int>] = []
    var start = 0

    for i in 1...count {
        var end = min(i * targetSize, file.size)
        if i < count {
            let newline = memchr(file.base + end, Int32(UInt8(ascii: "\n")), file.size - end)!
            end += file.base.advanced(by: end).distance(to: newline) + 1
        } else {
            end = file.size
        }
        ranges.append(start..<end)
        start = end
    }

    return ranges
}

Fanning the chunks out across a task group and merging the per-chunk results:

let ranges = chunkRanges(in: file, count: ProcessInfo.processInfo.activeProcessorCount)

var merged = await withTaskGroup(of: Results.self) { group in
    for range in ranges {
        group.addTask { processResults(in: file, range: range) }
    }

    var merged = Results()
    for await results in group {
        merged.merge(results) { a, b in
            Entry(
                min: min(a.min, b.min),
                max: max(a.max, b.max),
                sum: a.sum + b.sum,
                count: a.count + b.count
            )
        }
    }
    return merged
}

This episode uses Xcode 26.2.

All right, welcome back. It's been a minute, but we're back to tackle more of the 1 billion row challenge. I wanted to quickly talk about what happened last time. We had a couple of interesting findings. One of them is that we were misinterpreting the benchmark numbers here. I have in my Mies task, I have a time that was just scoped to the like time and then run the actual Swift release binary. And then Mies outputs like its whole task run here. And we were just sort of interpreting that as like, Oh, why is it taking 1.3 seconds? Like, even if we already built and that was puzzling, probably very frustrating for people who spotted that at the beginning. I didn't spot it until I was editing that episode. And I'm like, Oh my God. Uh, so, uh, so that is kind of, and this is a running it on a low, uh, like on a smaller set of data. I forget what the thing was, the 10 ,000 rows or something. Or maybe a million. Um, and that runs in 0.19 seconds. And then, um, I did a, um, a separate sort of thing to hopefully help us get more realistic numbers, like run it more than one time, do a warmup pass, et cetera. And there's a tool, uh, called hyperfine, which can do that. And so I build for release and then I run hyperfine with one warmup run. And then it does 10 passes on the same, uh, command and then gives me the time and the range of time, et cetera. So we're, we're at 50.7 seconds plus or minus 0.3 seconds. And, um, and that was consistent across the 10 runs. So, um, so we're getting much faster, right? I think we started at like 13 minutes or something like that. Um, with the, with the naive solution. So we're getting much faster. Um, so that is, uh, one thing to catch up on. The second one is where we ended up last time. I had a bug here. Um, when we were printing out the formatted, uh, temperatures, I had percent percent here, which is going to output a literal percent instead of the percent dot one F, which means that this one would have occupied the min, sorry, the max slot. And this one would have occupied the mean, sorry, this would have been min, that would have been max. And then mean would just be dropped off at the end. So, uh, if I get rid of that, um, we'll print out the values correctly. Uh, then more critically, we had an error, uh, down here where we were, uh, uh, grabbing these, uh, semi pointer, new line pointer. We're starting with the, the position that we're on. That's the city start. And then we have the city length. And then we reset our position to city length plus one. This should have been a plus equals to advance the position past the city. Right. Um, right. Cause we were starting at city start and now we need to go past the city, past the semi colon. And so that was a bug, which was causing a crash, um, on the larger data set. Um, and actually no on the smaller data set as well. Um, so I fixed those two things and that's, I ran the results and that's what we saw just then. The other thing we found. Line one 10, line one 10 is okay. It is a plus because we're offsetting by a length. Whereas line one 14 is acceptable because we're moving from an absolute and we're assigning with an absolute index. Yes. And I think this is really important not to confuse, uh, these integers, uh, they're both integers, but one is a length. The one is an index. And then you also have the pointer, right? So pointers indexes and lengths are all different things. Yeah. So just have to try to keep that straight in our, in our heads. Um, okay. Then the other big thing is we spent the whole episode talking about, Hey, Simdi is awesome. It's kind of this advanced thing that could take advantage of the hardware. Let's try Simdi to find the, uh, the things in the, in the bytes that we're looking at. So, you know, span across multiple cores and then say, are you a semicolon? Are you a semicolon? Are you a semicolon? Like 16 times at a time. And in theory, you know, that should have sped things up instead of just linearly scanning. And in practice, it didn't make it faster. In fact, it seemed a little slower. And, uh, that was puzzling. And so I did some research, uh, after that to say like, why, why did this not pan out? I was looking for a solution. And, uh, the reason is twofold. Um, the, the first one is, I think that this would be, um, a much better tool to use. If we were going through pages and pages of chunks, say like, are you a semicolon? No, no, no, no, no. And it's just like speeding through the data and eventually gets to a yes. And at that point, now we know that this 16 range of 16 bytes, one of those has the thing that we're looking for. And so we can linearly scan from that point onward. But in our case, our data is so short and every single line has this. So we're basically getting, using SIMD to say, yes, the line has a semicolon, which we kind of already knew. And, and then paying for that linear scan anyway. So it was like, you know, we, we almost never would go through a SIMD block that said, no, there's not the value in there. And so, so it just wasn't a good fit for it. And then the other aspect of it is this, um, memchar function, which is from libc, uh, which takes a base pointer to start with, a byte to look for, and a number of, uh, elements to stop searching. So we can say like starting from this pointer, look for this byte and go say 10 spaces. And if you don't find it, you return null. If you do find it, you return the pointer to this byte.

And so, you know, we basically can't beat this performance. Um, so those are the, that's kind of where we are. And, um, I think the SIMD exploration was interesting, uh, not only because it's an interesting technology, but also because we realized it didn't pan out for this problem. So, um, not, not everything is going to be a win. Um, okay. So today I was thinking we would finally tackle parallelizing this work using async await. Um, before we start, do you have any thoughts on how to approach this? I do. And in fact, I had this, when we first started working on this, I don't remember when we got to this process results shape of function, but it was pretty early in the series. And right away when I saw it, I was like storing it away. Okay. I have some thoughts for the future because what we're doing, where did that function go? Process results is one of our, that's like one of our core. Yeah. This thing here. So what it's doing is it's taking in this buffer and it is mutating this result structure and we can totally paralyze this, but it's a harder approach. Like if we had, if the shape of results was take input block and output, like we have a return value. Yes. This immutable result. That is an easier, probably an easier thing to optimize. Yes. We're not optimized and easier things to write. So if we just said return results here, right? Exactly. Yeah. So now I need to have a results up here. That is results. And then here I just return results everywhere.

Turn results. So this is actually pretty easy refactoring to do because now I don't actually need this one. I can just say let results equals that. You still, oh yeah, you're right. Is that true? Yeah. So we just have one. Yes. You're right. Do the one chunk, right? Do the one chunk. So now, so now let's call this not on the buffer, but like on the buffer with certain ranges or. Yeah. Yeah. We're going to split up the buffer now. Yeah. Okay. So maybe the, the easiest way to do that would be how about you have a, um, like a start and an end, like a range of bytes that you can operate on.

Yeah. And, um, so, and then we also need this to be sendable, right? Because if the data that we pass to this process results is being, is being assembled in a task group, then the, then the data that we pass to this has to be sendable. Yeah. And so the result output, maybe not necessarily. Maybe. I think what we're going to do with results though, is we're going to have many of them. And then at the very end, we just merge them. Right. So what does that make sense to me? Let's say we have a function called, um,

chunk ranges and chunk ranges is going to return a range of ints. Uh, sorry, an array of ranges of ints. So like sort of start and end into the file. Okay. And for now we can just say like, we rewrite it and refactor it like this and just have the one chunk for the whole thing. Right. Just so we make sure it works. So each range has, is a start and an end. Is that right? Yeah. And so we need to pass in that, uh, the file size and the, uh, what else does it have? The base pointer of the file. I guess so. Yeah. And so what I'm thinking is that all of this stuff can kind of like be included into a struct that we call a mapped file. And mapped file is going to be that base pointer, which will be an unsafe, um, raw pointer, um, and the file size, which will be a int, I guess. Okay. And then we can have a window into the bytes, which is going to be an unsafe pointer of uint8, which is going to be based on assuming memory bound to uint8.7. Yeah. Getting good at this, uh, unsafe pointer stuff. I'm getting better. Yeah. It, it, for a while it was just like unsafe, blah, blah, blah, blah, blah, blah. And, and I, over time I've gotten a little bit, I guess more, uh, familiar with it. Yeah. More practice. Yeah. The mutations is much more complicated, but we haven't had to do that. So that's good. Okay. So, uh, I want to say that this is unchecked sendable, which should be fine because that's not mutable.

Right. It's not going to. It's a teeny bit more complicated than that, but yes, in this case it is, it is immutable and also it points to only immutable data. So then it's fine. Yeah. Okay. And then, um, okay. So, so now I'm going to take that mapped file. Um, and I'm curious if I should make the init in here. Maybe I'll do it in one, uh, one step or more than one step. So we, we have that base pointer, which is the pointer and the size is the size. So we have this mapped file. You know, interestingly, I don't know that this matters and we can maybe mess around with, um, benchmark it afterwards, but I suppose it could be that this M advise is not appropriate. It will not be appropriate. You're right. Yeah. We've read in parallel. It'll be, uh, maybe, maybe the operating system is smart and be like, no, no, no. I can tell you're not reading it sequentially. Cause it is a hint, but yeah. It's a hint. Yeah. And it knows that, um, apps hint wrong all the time. Anyways, let's keep going. But I just noticed that and it's interesting. Yeah. Okay. So we no longer need this buffer because it's basically what we just put in this mapped file. Um, except we need a, we do need, is that what's inside of the, uh, map file type? You put an unsafe buffer pointer? I put an unsafe raw pointer. Um, okay. Maybe it doesn't matter. I think that we will convert it to a buffer pointer inside the, inside the process function. Like basically we want, we want to say like, given this file and a range, then you can extract your buffer. Make our buffer from there. That makes sense. Yeah, that makes sense. Okay. So, uh, process results now, uh, or sorry, uh, we need to get our ranges, which is going to be chunk ranges. And I want to pass in the, uh, the file. And then how many ranges do we want? Um, at this point I could just say one.

And so we can say in file mapped file. And the count is in it. So the, um, what we want to do, let's say the count is like four. We want to take the size of this and divide it by four roughly. Right. So we're going to have like a target size, which will be an int and that's going to be the file size divided by count. Right. And this is integer division. Um, and we also know that if we divide the file by let's, let's say we have like, um, you know, some city, semicolon, some temperature, uh, and then a new line. And let's say that divided by four puts us right here. Yeah. We need to like walk ahead until we find that new line because it has to end like it has to have a clean, uh, new line aligned chunk, which is a term that I think we used early on new line aligned chunks. New line aligned. Uh, okay. So let's get our ranges, which will be our return value, which will be the range of ints and we'll return it at the end. And then, um, here we're going to say, uh, for I in zero to count, uh, less than no.

Yeah. Zero to count. And we want to take the, um, so this is going to be our, our chunk.

Index. So we need a start and an end. Yup. Um, our start is going to start at zero.

And then we need an end, which can be if we are at the end of the file. Uh, so we're going to do I times target size, which that's going to end up being zero. So I could do I plus one times target size. Um, or maybe just do one, two, including count I times target size. And then I want to make sure that I don't walk off the end of the array. So this is going to be filed up size. Uh, the end of that. That is, okay. This is true, but I don't think this is taking into account the previous, I think start has to be taken into account here. Um. Oh, you're just computing end right now. Wait a minute. Sorry. Yeah. We will mutate start to be start equals end at the end of this loop so that we start from the next spot.

But I, no, no, I see what you're saying. This is, this is working. This is going to work, I think. Um, if we're on the last one, so if I equals count, then, then we know that the end is just the file size.

Mm-hmm. Which, which may have already been, but, um, if we're not, then here's where we need to like search ahead until we find a new line. Um, yes. Which we can use mchar to do. Yep. Or memchar. And this is the files base pointer plus that, uh,

start. No. Plus end. Where we, like the end of where we thought we were going to be. Yeah. But we can't do, we, I don't think we can do divi, um, can we do pointer arithmetic like that? Yes. We can. Yeah. File.base plus five gives us a new pointer that is five. Oh, okay. Offset. Cool. I didn't realize that. Yeah. So, um, what we're looking for is a uint8, uh, new line byte. And I think this needs to be cast to int32 because that's just the, what the function. Yeah. Signature is. And then how many, how many bytes can we, uh, look, continue looking for? Um, I mean, I, I'm inclined to say like, oh, infinite, but, uh, we, it would probably be naive enough to try to walk off the end of the file. So. I'm sure it would. I'm sure it will. Yeah. And then crash. So, uh, let's say that we, the maximum that we can go is to the end of the file. So that would be the file size, um, minus where we started from. Minus. Exactly. Yes. I think that's roughly possibly plus or minus one. Correct. Yeah. Uh, okay. So then, so now we have a new line and now we need to get where that new line is.

Um, we need to, uh,

the difference between file base plus end and where that new line ended up, that, that, um, starting, we want the difference. How, how many characters it'd be advanced, right? Correct. Um, we started at file base plus end and we ended up at this, this, um, new line position. Oh yeah. There's this distance to function where we can grab new line. Um, yeah.

So you're saying, um, File base plus end is where we started. And so it's a new line minus file place. Yeah. So you're saying file that base plus end minus new line. No, Uh, I guess it's actually, yes, that is exactly what I'm saying. Yeah. No, new line, new line minus. Uh, so this is the full index. Is it not just new line?

Oh, this is a pointer. Sorry. This is a pointer. Uh, yeah. New line is a pointer. Okay. If we are starting position was file base plus end. We've advanced somewhere. Where am I? There we go. We've advanced somewhere now to here. And that's that, that difference between these two points is a length. And we want to add that to end. Is that right? I think so. Yes. So, um, well, what I was thinking about this distance to takes a pointer, not a integer. And that's what I was trying to use a minus for. Okay. So because this is a pointer, it's unfortunately a unsafe mutable raw pointer optional. I'm going to assume that we're going to find one. Sure.

And can I just pass this to, yeah. Okay. We can pass a mutable pointer as an unsafe raw pointer. Um, okay. So that gives us the, the distance from the start of the file to this new line. That gives us our total window. But we want to be one past this. So that start starts at the character right immediately after this. Right. So we're going to like read up to end, not inclusive, if that makes sense. Okay. Um, that makes, that does make sense. Yeah. Okay. So then start equals end. And then we also want to say ranges dot, not insert, append and then get a range, which is going to be start to end like that. So not inclusive. Not inclusive. Yeah. Cause we stepped over one. We stepped over the new line. Okay. Yeah. This makes sense. I think this works. Um, and so now we have ranges, which should be the entire file. Um, and, um, if I pass in process results, this is now going to take, um, well, this is more going to be a range, but let's, this is going to take our file.

Um, mapped file and the range to use. One single range. That's right. Um, so our base is now going to be our files base plus range dot start, uh, start index. That's what, yeah, that's no analogy with range. Uh, no, lower bounds. Yeah. Not start index. Cause that, the range is also a sequence. Uh, yeah, yeah. Right, right, right.

Uh, count is going to be range dot upper bound minus range dot lower bound.

And honestly, I think at this point I can actually have a base, a buffer. I'm pretty sure you have enough information to make one. Right. Well, what I was hoping to do is just get this compiling. So let me see if I can, uh, so the pointer to the C. We don't need to say the base address is base. Uh, so we can just say base advanced by. I'm pretty sure. Uh, wait, base is now an unsafe raw pointer. It needs to be, yeah, I need to say, um, assuming memory bound to you into eight dot self here. Oh, make it, to make it the same thing as it was before. Uh, temperature bytes. Okay. This is now going to be. This is a problem cause you don't have a buffer. Yep. Uh. I don't think you can index into a, maybe you can though. No, I think you're, I think you're right. We would need to create a buffer here. The thing that I'm curious about is that our buffer didn't start at zero.

Um. That is okay. Okay. So if I, if I say let buffer is an unsafe, uh, mutable, no, unsafe buffer pointer. Yeah. And it's C start and count. So I start at base and count. You start at file. Yeah, you do. That's right. Yep. And everything else is. Okay. So what I, what I just wanted to do here is just set a break point here, run it and see what our chunk looks like. Okay. Um, so this is going to be in file and then we'll do ranges zero just to get it, uh, range just to get it, um, running. Yeah. And we're expecting the whole range. Yeah. So our, uh, ranges here is. Wrong. Uh,

is it wrong? It does look wrong. Is it? Oh, wait a minute. Is this, is this the whole, well, lower bound and upper bound. Yeah. I can't be right. It's, um, okay. Let's make sure that the file. Are we using the big file though? We are. File is 15. Okay. Okay. So this is, uh, let's look at, um, file.size is looks like correct. Right? So that's 10, 24, 14 gigs. Yeah. So that's fine. Then, uh, file.base address is going to be some pointer into memory. That's suspiciously even, but, uh, the, the, the last four digits are not surprising to be zero. Cause they pass a page align it. Yeah. So it does, it does, it does look weird, but it doesn't look that weird given that it's a memory map file. Okay. So that seems fine. Let's go into chunk ranges and see what we did here. It's probably an off by one error. I have a feeling. It looks like this is the total size of the file and we didn't start at the right spot. So, yeah, start. Oh, you know what? That's it. Aha. I assigned start before doing the range. Okay. So let's run this again. So we expect the, uh, start and end zero. Okay. Perfect. Okay. It worked for one. So now, uh, let's see if we can get it to work for, I don't know, four. Now they should not all be even because they're not going to be the same size, but we have zero to three, seven, eight, three, two, three. Which is correct.

Yeah, this looks correct. Well, it certainly looks consistent. Yes. Uh, would you like to examine the, to see if these are new line characters? I guess. How hard is that to do? That does seem worthwhile. I don't remember how we did that. Um, what do we, what kind of types do we have here? So we were going to do file that base plus, uh,

ranges one dot upper bound. Yeah. Gives us a pointer to that thing. Yeah. Um, and this pointer, I want to examine in memory. You want to do reference that. Yeah, exactly. If we take that and do point T. Yeah. Oh, we need to assuming memory bound to, uh, actually I think I can do C chart itself. Probably. Yes. Dot point T. Gives me an 84. Is that a new line? I think so. I don't remember, but it's believable. Okay. Hang on. I'm going to do. It looks ASCII. Uh, E print. New line. Oh wait. Uh, it's not gonna, that's funny. Uh, it's not going to print like I thought, thought it would. No, I don't. 84 is the capital, is a capital T. Capital T. So that seems wrong then. What is new line? Line feed. That's a, that's decimal 10. All right. So let's take a look at, so we're, we're off by one. Is that right? This actually does look right. Cause the one right before it is the new line. Is that what, that's what we want. So end is the new line plus one. And that, yeah. I don't know if we're going to include the capital T. The T is the start of the next city, which, which is the start here. So that makes sense. Okay. This is the T. This is the, this is the T, but we're doing less than or equal to the end. So it's the new line. So I think. Oh, I understand what you're saying. That does make sense. Okay. So now we have one process results thing that we can do, uh, many times. Now I want to split this up into a bunch of chunks. We have a couple of. Well, you know what we should do let's first, uh, results. Oh, do we need to, cause we only call process results once. We now need to call process. Let's make a, let's make a sequential loop here. And I think that'll help. So if we, um, like a sequence, like non-async you're saying. Like, let's just write a for loop right now and make this work with the number. Cause it, it right now it would only process the first range. Yeah.

Okay. Uh, range. And then here I, now we're going to have multiple results and I need these to be merged. So I'm going to do merged is a results. Mm-hmm. And then I'm going to say merged dot merge. Merge. Merge. Merge. Unique and keys with, and this gives me an A and a B entry. And I need to create a new entry with a min, max and sum because you could have cities spread across many of these chunks and we need to like, right. Yes. Yeah. So entry min, max, sum count. Why didn't that work? Option enter. Okay. Min is going to be min of a dot min and B dot min. Yeah.

I, yeah. It's close, close to what you want, but no. Not quite. There you go. Uh, I don't know how to tell Xcode to do what I want. Not what I say. Max, A dot max, B dot max. The sum is going to be, um, A dot sum plus B dot sum. And then the count is A dot count plus B dot count. Correct. I think this makes sense. Okay. So now at the very end, now we have merged dot map and let's just run it on the small data set, uh, this one and just see if we get the same answer. Yeah. Yeah, yeah, exactly.

So looks reasonable. Let's look, the last couple was Amuda was 10.7. Aina Zell was 14.6. Okay. And if I scroll way up, where was it? There it is. Uh, yes. 10.7. This looks the same. All right. So I like that step of doing it sequentially first to make sure that we're in working order before continuing. Um, at this point, we're not far away from, uh, just saying like, oh, I want to do this. That's why doing, making the sequential loop first is a useful tool also because the shape is going to be very similar. Okay. So it doesn't like the fact that, uh, why doesn't it like that? Because the results has, the results is a type that has a key type that is not sendable. Okay. So that should be easy to fix. Let's go, let's go look at that. Um, our city key, um, and if you remember the city key in order to, uh, to avoid creating string keys over and over and over again, we point to the, uh, we point to one of the keys. Yeah. Right. Cause there may be duplicates everywhere in this file, but we point to one of them and we store the count and we pre-compute a hash to it. We get a consistent hash because we pre -computed the hash to it. And when we are comparing to, for equality, we make sure that, uh, the count is the same and all of the bits are the same. Okay. But we don't compare the pointer because the pointer is not going to be shared, right? It's going to be many pointers, but we can choose any one of them as long as they point to the same data. Uh, this is going to like the, the memory map is going to be alive the entire program. Yeah. And we're not writing to any of this data. So I think we can also do unchecked sendable here. We can. And what, the thought process you went through there of, cause many people look at immutable data and they think, Oh, has to be sendable. That's like sufficient for thread safety, but immutable data can also point to mutable data and then it's no longer thread safe. But in this case, everything is immutable. So we're fine. Yep. I have often had that exact thought. Oh, it's immutable. All lets. We should be fine. Yeah. Especially if that let points to an NS object of some sort. Exactly. Yeah, exactly. That got rid of our error. Uh, now this instance method that requires a task results never conform to sequence.

Um, you put, you, you put a task in. Yeah. But I think what we want to do, I think what we want to do is spread this workout basically like effectively do like a map and reduce across all the ranges. Yes. And the way that we can do that, there is a tool in Swift that exists to do exactly this, which is called a task group. So that's a with task group. Exactly. There's a couple of different ones, right? With. Well, there's a, there's a throwing, there's a version that throws.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know. I don't know.

I don't know.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know. I don't know.

I don't know.

I don't know.

I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know. I don't know.

I don't know. I don't know. And then these will come in in the order that they just arrived. Right. So it's like when group. We'll get one. Okay. And we're saying four right here, which, you know, I have a Mac Studio M4 Max. So this has, I don't know how many cores this has. A lot. And there is, we could get our cores from process info. Process info dot core count or something. Probably. There's something like this. Processor count. There you go. So that will give us cores, but I am curious, like some of these are efficiency cores and some are performance cores. And so the performance cores are probably going to like win and just be waiting. So we may decide like to make this number bigger so that there's like a pool of work and then the cores just chew through them in the, you know, as much as they can. Yeah, I mean, I think what you want to do. So if we made count a very high number, let's say, like basically we tried to process every single line in parallel. That's a very large number of tasks that we're creating. But as far as the runtime is concerned, that's the runtime's job is to schedule these things efficiently and to make use of the hardware as appropriate as possible. The disadvantage of doing something like that is we make a lot of tasks. There's a lot and those have a memory cost. Okay. Whereas the reverse is we make a very small number of counts. It is conceivable to me that processor count is a suboptimal number for reasons like what you just said. It's hard to be sure that process count is the right thing to choose. So I would start with like not thinking about it that hard and then we can play around with it a little bit and see if it makes a difference. So you put four. I think four is a totally fine place for us to begin. Let's start with four because I actually just would like to see this. I'm pretty sure I have BTOP on this machine. Yeah, I do. And so this is actually going to show us. It's like top but nicer. Yeah, okay. Like it's graphical, right? So we have this, you know, my CPU's cores and stuff. You can kind of see what's happening. And obviously I'm doing video encoding while doing this. We're recording and all this stuff. So it's, you know, this will probably run better when I'm not recording. But I will run that here. And then we're going to, I'm going to run just the small one because that happens pretty quick. Yeah, yeah. Actually, no. The big one took like a minute before, right? It was like 50 seconds. Okay, let's go for it then. Yeah. Let's go big.

Okay. And it looks like cores one through three are doing stuff. So you have some idle cores, but that could potentially be an impact of how many we decided to use. Right, we said four. There's more stuff. But look, they're all not that busy also. Yeah. I mean, there's some IO happening. Yeah. There's no way we're going to be able to. Also, we did forget to remove the MAdvise sequential. That took 38 seconds down from 50 seconds. Okay. It got faster. So it got faster. Not four times as fast, but I think that is not to be expected. Right. No, not to be expected. Yeah, for sure. So let's, I want to try this to no longer call this to see if that makes any difference. Yeah, that's an interesting, this is a very interesting change. So the cores are marginally more occupied now, potentially, which is, I think, also, interesting. Oh, that was a big change. 13 seconds. Wow. Yeah. I also didn't warm it up before. So maybe that has a factor, right? Because in the hyperfine benchmark, it doesn't matter. That will matter. Okay, run it again. Run it right now, just one more time. Do you want me to flip it? You haven't done anything else, really? Do you want me to flip it back, though, to the sequential one? You could. So what I could do is do this and then run the, because this isn't going to take very long, we can run the benchmark. Actually, let me go in here and see.

Hyperfine has a number of runs, and I can reduce that.

Warmup, min runs, max runs, runs. Okay, so dash r and dash w. So we're going to do, these are benchmark, warmup one time, and then runs, we'll say, run it twice. Sure. And this is going to do it on the big data. And this is with mAdvise sequential, and then I'll run it again without and see if that did, in fact, make a huge difference. It is nice to be able to run the benchmark on the whole thing now, because it's not. The many benefits of having a reasonable data set to run on. Yeah. Or reasonable program execution time. You can see that my memory for this is hovering around the size of the file, which makes sense. Now, I think it's interesting now, all of a sudden, all your cores are busy. That surprises me. I think that's just because maybe the scheduler for some of these other things, like video encoding. Something else just could have come up at the same time. I think it's going to be like, oh, I'm going to pick a core to do this work on. And these are busy. So maybe those works are being spread across other cores. Yeah. So this was the mAdvise version, now took 13 seconds, which is what we had before. Yeah. So I think it was the warmup that was costing us the 38 seconds or whatever. Yeah. And this didn't really make much of a difference. Let's run this one more time. And comparing 13.897. You know what? It's interesting because the first thing we do is we scan the file and make chunks. So we do use this memory sequentially to start. But then we go back to not really. Well, kind of. We kind of do. It's not really. Yeah. We jump to a specific point in the file and then go sequential. Right. It's not sequential. Yeah. It's not sequential.

Okay. 13.99. I think that makes no difference whatsoever because of all the other factors happening on my machine right now. So I'll leave it commented out because it's an interesting point. Now, what happens if we choose a much bigger number? I mean, let's see what happens with cores. But my suspicion is that if I run with like cores times 2 or something like that, that it will be faster than this. 100% utilization now. Yeah. And I wonder if the recording is going to be affected. Oh, and look at this. But estimated time is five seconds. Oh, yeah. Much better. Much better. And yeah, you can see it light up the CPU here. Yeah. So now let's try. I'm just curious. Which is just amazing by the way that my fan is not even on. I can't. I can't hear the fan on the screen. If you kept that up for a while. If you kept that up for a while, we probably would hit it. Let's try it. Just for giggles. Let's try cores times 2.

So my theory is that this is more or less going to be roughly just overhead and we're going to get a worse performance here.

So it got a little faster. Yeah, slightly. I don't know if that's significant or not, but yeah, this is one billion rows in five seconds. Is that good? How is that compared to other results? Is that good? This is a great question. Let's see.

Yeah. So one and a half seconds is the fastest. Okay. So we're on the right order of magnitude now. Yeah. And I do remember taking a look at some of these solutions just to be like, what does it even look like? It's worth taking a look because it is not code you would be like, I want to write this code. Yeah. Let me find an example. I'm not sure if this is the one, but there are some of them that are like, it'll do things like this to avoid branching. Like, yeah, it's avoiding branching and it's avoiding duplicate work, which we're not doing. And there's all kinds of spots where we're doing duplicate work. Yeah. So like, there's all these like, oh, I know that there's like, oh, delimiter mask one, two and three. Like this is code that I would never do normally, but because it can just do like one, two, three sequentially and not have a conditional, then the CPU just blazes through the instructions. It doesn't have to jump around. Yeah.

Yeah. So things like that, you know, are interesting to look at. But I think this is great because we're at an order. We're on the right order of magnitude of a good result. Five seconds is definitely not on the leaderboard, but it's not bad. But we haven't put in considerable work. I mean, I think that if we return to span, our pointer arithmetic would be much simpler. Yep. And in comparison to how much code we had to write as well. Yeah. Yeah. That payoff was pretty nice to see. I think that, you know, in the next one, I would like to investigate the span refactoring and then take a look at instruments just to see where is the time now, because there's clearly some more gains to be had. I don't think I want to go to the level of, you know, optimizing the branchless code output. No, me neither. You know, for marginal gains or whatever. But I do think that there's probably a couple of other small pieces we could tackle and then call this done. Yep. All right. As always, thank you so much for joining me and we'll see you again in the next one.

Source Code

View on GitHub Download Source

Series