Scott Keck-Warren
0:04
Hello developers, and welcome to the PHP Architect Community Corner, where we have conversations with members of the web development community. I'm your host, Scott Keck Warren, and today we're talking with Florian Engelhardt about concurrency inside of PHP. Florian Engelhardt is a proud dad of 5, a coder, an author, a speaker, and a geek. He currently does PHP profiling at Datadog. Thank you, Florian, for finding some time to talk to us today. Thanks for having me.
Hi, Scott. Thanks for having me on the stream.
Scott Keck-Warren
0:29
Yeah, absolutely. So I know that— so the reason that we're having you on today is that you wrote an article in the October 2024 issue of PHP Architect about concurrency. Can you talk to us a little bit about what concurrency is and how to use it kind of in PHP?
Yeah. So generally concurrency is just that you are doing multiple things at the same time. That's basically what concurrency means. And usually people tend to do that when like the most normal setup that people have is PHP-FPM. And then you have like, by default, 4 processes that concurrently process incoming requests. But this is something that is— how to say that? Like, you're not doing this like explicitly. This is just happening implicitly because the PHP code that you're writing is not meant to be run in parallel. Or it's not like you're not doing this explicitly in your code, but Like the surrounding environment supports this. And when, but actually in PHP, what you can do is you can explicitly write code that is concurrently, like your PHP code being concurrently doing multiple things at the same time.
Scott Keck-Warren
1:41
I'm a university-educated traditional programmer. And so the way I always look at code is it goes like, it does A and then it does B and then it does C and then it does E and then it does F. But like, What we're talking about here is like, so like if I'm making cookies, like with my kids, then like I might start like mixing the wet ingredients and I can have my oldest go over and wet the— mix the dry ingredients. And that's like the concurrency we're talking about here where we have like the same processes occurring, but then somehow we come back in the— at the end to like mix them together.
If you're baking cookies, the, uh, well, the one view could be like you're doing preemptive multitasking. You are doing one thing. And then use like another person's doing some other thing while you say like to your daughter, probably do this, or could you do this, or could you do that? You're not actively waiting for that. You're not blocking, but you are doing something different. And then later you're gonna check the promise if it actually resolved, like if your daughter is ready doing the thing.
Scott Keck-Warren
2:41
Yeah, 'cause otherwise like I'd have to like wait until she's ready with the thing. the wet ingredients to then mix them all together.
Yes, you would just stand there and wait. But you can do something different in the meantime because you're not the one doing something.
Scott Keck-Warren
2:53
Right. And then, so then, so then are there other types of multitasking that we could, we could do inside of PHP other than, than that type, than the cooperative multitasking?
Definitely, there is preemptive multitasking where you would spawn multiple processes or multiple threads to do multiple things at the same time. It doesn't really fit to the cooking example because it would be you as a person doing multiple things at the same time and people are bad at multitasking, but a CPU is not.
Scott Keck-Warren
3:18
Yes, I'm terrible at multitasking. I'm fairly good at like single tasking sometimes. So, because I think like my, my like ultimate like entry level into, into like doing like multiple things at the same time has just been like queuing stuff, right? Like I like, I have like, I'm making my cookies And then I also go, oh, can you clean up? Like, right? Like, that's the other— I can do all of the cookie recipe, but then like ask somebody else to like clean up my mess as I go along. That's like another like topic that you talk about inside your article is like sending stuff into a queue, right? Is there a term for that as well or?
Well, technically that would be just queuing. The idea is then like if you send it to, let's say, RabbitMQ, Kafka, whatever queue system that you have, If you consume the messages from there, then there's multiple ways of doing the thing. Like you could just process one message at a time. Depends a bit on what you're doing in processing the message. So if you're processing the message and that's like send tracking data to a third-party service, which is basically just an HTTP request probably, then you would spend— like your worker would spend the most time waiting. And you could use stuff like ReactPHP or mPHP. to not wait on the result of those HTTP requests, to just like, you get the message from RabbitMQ, for example, you send out an HTTP request to the third-party tracking service. And in the meantime, you fetch the next message while you're waiting for the result. And then later you just check, is there a result? No, there's no result. So let's fetch another message from RabbitMQ and just hope that sooner or later there will be a result. If the messages that you're processing, the problem is CPU bound, then ReactPHP or AMPHP might not be the solution, but probably more something like Swoole or XParallel, which allows you to start multiple threads. And then you could, you could have one thread that consumes the message from RabbitMQ and sends a closure to another thread to actually do the work on another core on your CPU. And whenever that work is ready, you can send the, the result back to RabbitMQ or whatever the your message wants you to do.
Scott Keck-Warren
5:27
Okay. And so, and there was a lot of terms in there and like, I'm not the best at these. So like, so RabbitMQ and Kafka are both like, are other services that you'll be running somewhere like in your environment. And then Swol and React and I forgot the third one. Those are all libraries that are built on top of PHP?
Yes. So to a certain degree, ReactPHP and AMPHP are 2 userland frameworks that you can use to do asynchronous software development. Both come with an event loop implementation. And there's a lot of libraries around that, that allow you to do asynchronous stuff in PHP. I know a bit more about ReactPHP than about AMP PHP. So ReactPHP, I think support goes down back to PHP 5 point something. Should probably not use that, but it would work. So if you are somehow locked into PHP 5 or PHP 7, ReactPHP would work. to get you an event loop implementation. And I know from MPHP that it's using the Revolt event loop implementation, which is baked by Fibers. So the nice thing about MPHP is that you get a, let's say, natively looking synchronous code that behaves asynchronous. I think you need to read the article.
Scott Keck-Warren
6:52
Yeah, no, totally. Every— and I'd say like everyone should go read the article. It's like really very helpful. 'Cause like the, the best example that like, 'cause I'm, I'm very much like in my mindset, I guess, for, for lack of a better term, is like, I'm, I'm very good at understanding, okay, I can throw this out to a queue and then that can get handled independently. But then like when we start dealing with like multiple flows of like coming, like the same process, having multiple things going on at the same time, that's when it gets fuzzy for me. I'm always just like, well, how is this working in reality? And the one really great example that you have in your article is you're requesting 2 HTTP endpoints at the same time. And that's something we do at work. We have a thing where we have to do 3 API requests and they don't have to be done sequentially. They could be done in parallel. And because we're doing them sequentially, they take 5 minutes. But I think if I do them in parallel, it would take like less than 2 because one of them returns like just an insane amount of data. So that was like, to me, I was like, oh, this is how I can use this at work, like immediately, like in your article. So that was really helpful.
Yeah, I mean, the main thing I wanted to express with the article is that in most cases, when I talk to people about PHP or about like handling multiple things at the same time, it's like, oh yeah, I can use like, I have PHP for my web service and I'm gonna Uh, send this to a queue and then we need to, uh, oh, we need to process the stuff. Um, uh, so probably use Node.js or Go or any other language. And I'm like, you can do this with PHP too. The solution is there. And I think it's also, um, like it's never a bad idea to learn a new language because there's new concepts and you're gonna learn a lot of new stuff from them. Um, but also, um, it's easier, I think, to learn a new, um, programming paradigm like asynchronous PHP in the environment that you already know.
Scott Keck-Warren
8:40
Because then you're not learning multiple things, right? You're just learning the one thing. Yes.
You know, you know how to use Composer to get your dependencies, you know, to write your tests. You have your local environment, development environment set up for PHP. Everything is there. So that is helpful. That's a lot of problems that you don't need to solve switching to another language. And you can learn this paradigm and then you maybe have an easy entry into another language if you want to learn that.
Scott Keck-Warren
9:04
Okay. That all sounds really interesting. Again, like, so it's in the October 2024 issue of PHP Architect. It is a really nice read. It's very easy to read, even if some of the concepts are a little like hard to grasp essentially. And that's my fault, not Florian's. So is it like, how is it that you use concurrency like in your day job?
So in my day job, I'm building a profiler for the PHP language. And this profiler collects stack traces. Now, in order to do so, we need to stop the engine, and then we can collect the stack trace. Otherwise, if we would not stop the engine, the stack trace that we want to collect would be a moving target or would be just gone by the time we try to collect it. So we need to halt the engine. Now, when we halt the engine, this means that your PHP script will not execute. It will just wait until we are done collecting the stack trace. Now, there is other stuff that we also have to do. And that is, for example, the stack trace that we collected. We need to aggregate the data in memory, and then every now and then we're going to upstream the data to the backend. Now, aggregating the data and upstreaming the data or serializing it and then sending the profile upstream, you don't necessarily need to wait for this. Like the PHP process doesn't wait for— does not need to wait for this. So what we are doing is we halt the engine, collect the stack trace, and once we have that, we send this sample information to another thread. that then does the aggregation and serialization and upstreaming the data while the PHP process can go on and you don't need to wait for all of this to happen. The only thing you need to wait for is to actually— the stack trace collection. After that, the PHP process can go on while asynchronously the other thread does the aggregation, serialization, upstreaming, stuff like this. Yeah.
Scott Keck-Warren
10:50
Is there anything else you want to talk about concurrency right now? Or I think we're pretty good on that.
We could talk about XParallels.
Scott Keck-Warren
10:58
Oh, okay. What's XParallels?
XParallel is also in the article. There is an example in there. The— no problem. So XParallel is an extension that's written in C for PHP, and it allows you to do threading in userland. So you can start multiple threads from PHP. And then give them tasks to do. And you would use parallels if you have a CPU-bound problem. Like the examples that we talked about, like sending HTTP requests were I/O-bound. You just wait for some third party to give back a response. But if you have a CPU-bound problem and you want to utilize multiple of cores that you have in your machine, then XParallel could help you because it allows you to start multiple threads from userland.
Scott Keck-Warren
11:50
So, is there anything else that you'd like to direct our listeners to, like either your website or your Twitter or whatever before we say goodbye today?
You can find me on dotbox.org. Otherwise, I think you have the Mastodon handle.
Scott Keck-Warren
12:05
Yep. And we'll link to those in the description. So, awesome. So, thank you for your time again today. And for those of you who, if you ever go to a conference and he's— and Florian is speaking on how to do testing in PHP, it's a really fascinating talk. I was lucky enough to get to see it last year at Longhorn. And it is like very fascinating, like how the test structures essentially are built. And it seems like something that anybody who is a normal PHP developer could kind of help out with. So I highly recommend that. So thank you. Have a great—
Thank you very much for having me.
Scott Keck-Warren
12:38
Thank you for listening to our podcast. If you're not already a subscriber, make sure you subscribe with whatever your preferred method is so you can get our episodes as soon as they're released. Maybe share this post with a coworker so they can learn from it as well. This podcast is available both as an audio version on the PHP Architect website, as well as a video version on our YouTube channel. This is Scott Keck Warren for the PHP Architect Community Corner signing off and reminding you to keep listening, keep coding, and keep reading.