In this episode, Scott talks with Andrey Kucherenko, creator of jscpd, about why copy-pasted code is an anti-pattern, how jscpd detects duplicate code across dozens of languages, and how the tool has evolved to work inside agentic coding workflows.
Links:
- Our Discord - https://discord.gg/aMTxunVx
- Buy our shirts - https://store.phparch.com/products/community-corner-podcast-t-shirt
Andrey's Links:
- jscpd - https://jscpd.dev/
Scott's Links:
- Website - https://scott.keck-warren.com/
- Bluesky - https://bsky.app/profile/scottkeckwarren.bsky.social
- LinkedIn - https://www.linkedin.com/in/scott-keck-warren-91689810/
- Mastodon - https://phpc.social/@scottkeckwarren
PHP Architect Social Media:
- X: https://x.com/phparch
- Mastodon: https://phparch.social/@phparch
- Bluesky: https://bsky.app/profile/phparch.com
- Discord: https://discord.phparch.com
Subscribe to our magazine: https://www.phparch.com/subscribe/
Partners
This podcast is made a little better thanks to our partners.
Displace
Infrastructure Management, Simplified Automate Kubernetes deployments across any cloud provider or bare metal with a single command. Deploy, manage, and scale your infrastructure with ease. https://displace.tech/
PHPScore
Put Your Technical Debt on Autopay with PHPScore https://phpscore.com/
CodeRabit
CodeRabbit - Cut code review time & bugs in half instantly with CodeRabbit. https://www.coderabbit.ai/
Music Provided by Epidemic Sound https://www.epidemicsound.com/
phpc #php #communityCornerPodcast #podcast #jscpd
Transcript
Scott Keck-Warren
0:05
Hello developers, and welcome to the PHP Architect Community Corner, where we have conversations with members of the web development community. I'm your host, Scott Keck-Moren, and today we're talking with Andrey Kocherinko about JS Copy Paste Detector. Andrey Kocherinko is a chief software engineer and architect with 24 years of experience building and shaping software systems. He created jscpd.dev, an open source copy paste detector, and is currently building gangsta.page, a project focused on bringing discipline to AI-assisted coding. Outside of work, he is a father of 3. Thank you so much for finding some time to talk to us today.
B
Speaker B
0:45
Thank you. Thank you for the invite. And let's start to talk about technologies.
Scott Keck-Warren
0:51
Yes, absolutely. So let's start out kind of at the very beginning and let's talk about why copy and pasting code is bad. and why we shouldn't be doing it as developers.
B
Speaker B
1:00
So let's imagine the situation that copy paste is not bad and I copy paste everything everywhere. And in case of changing something or apply some new features for code base, I should go to each of the piece what I copy pasted and edit it. And the agents should do the same. If we work in the team, my colleagues can hate me in case of a lot of copy-paste they should change. And so in general, it's like optimization of work for every developer, for every agent, what we— who should use, read, and write code together with me or together with other people. And so I would say it's like anti-pattern of code writing and code support and maintainability of the code. And also we can talk about clean code and in Clean Code book we can find pieces that copy-paste is one of the bad things what we should avoid in our projects. And so I think it's pretty straightforward and understandable that copy-paste, it's not good. But yeah.
Scott Keck-Warren
2:34
Yeah. And I think like from my kind of perspective, like copy and paste is bad in general, right? Like if we're copying code from one section and pasting into another, that means like we have 2 copies of the same code that we have to maintain, right?
B
Speaker B
2:47
And so we support both parts, both pieces of code and spend time for that. And that's why it's not good.
Scott Keck-Warren
2:56
Right. And then, so, so let's talk about like, so the project that you're working on, uh, or have been working on for a very long time, um, is JSCPD and, and it kind of solves, helps you find like these, these problems. Can you talk to us like how, how JCP— JSCPD does all of that and, and what, how it helps?
B
Speaker B
3:14
So I was inspired by project from PHP Vault. It's named PHP CPD. It was created by Sebastian Berman, author of PHPUnit. And for now it's abandoned project, but I review it before I start my version. And I faced this situation that on my project a lot of copy-paste were in HTML files, in Java files, and in JavaScript files. And PHP-CPD work just for PHP code. But I need to solve this issue. I need to improve my pipelines and add a reporter, add a highlight for copy-pasted parts of the code on continuous integration level. And I tried to investigate and didn't find any a proper tool for that, and I decided to make this by myself. I investigated code of PHP-CPD. Also, I reviewed code of this huge project named PMD-CPD. It's like for Java and written on Java. I found that this tool used both of the tools use Rabin-Karp algorithm to detect duplicates. Also, I find a couple of other interesting algorithms, and, but this Rabin-Karp algorithm was the best for my case. I implemented in JavaScript. To be honest, it was not JavaScript at the start, it was CoffeeScript. After that, I moved it to TypeScript and now a new version rewritten on Rust. It's extremely fast and supports a lot of languages. And how I did this support of different languages, I need to make from code some set of tokens. And in JavaScript, we had a lot of projects that need to do the same but for other reasons. And I get some code highlighting tool, extract tokenization, tokenizer, and use it inside copy-paste detector. And in this case, I can transform all different type of codes written on different languages, transform to set of tokens, can filter tokens, can remove some comments or some useless things from the token list and compare different files with RobinKarp algorithm and get results. And I do this in Rust as well. You can see how it works. gscpd-dev.
Scott Keck-Warren
6:45
Yeah. I, so, and so the reason that I found like the project is like, I have been using PHP Copy Paste Detector for longer than I should probably admit. Like it's, it's been a long time and it's not being updated anymore, but it does work. Like it's a functional thing. Like it's, it's kind of like one of these tools where it's just like, we don't really need to add more to this. It's just kind of does what it does. Um, but then we had like somebody came along and they're like, well, you can't have this package in your CI/CD pipeline that's no longer being maintained. And so that's when we started looking for alternatives. And, um, and I'm glad I did because I came across like your project and I am just like, I cannot tell you how flabbergasted I was when I like ran it and it like returned the results and it was just under a second. It had like done tens of thousands, hundreds of thousands of lines of code and just like, it was like less than a second. And I just like, was like, nope, I did something wrong. This isn't checking for what it's actually checking for. But then I just like fiddled with it and it's, and it's actually working. So it is, shockingly fast for anybody who has worked with PHP copy paste detector. So I think, I think everybody should know that if they are, it's well worth the upgrade.
B
Speaker B
7:48
Thank you. Thank you for the feedback. And I tried to, to make it faster than other competitors. To be honest, I was pushed by competitors and I've found some projects, they do the same, but for other languages, for other algorithms and try to understand how to speed up my own project. And I decided to rewrite it to Rust and make some benchmark, compare with different tools. And you can find it in GitHub repo results, probably 6 or 7 different tools. I compare with copy-paste detector and improve continuously and try to make the best performance. And for now it's the best performance for all, between all of the tools what I collected that can make this copy-paste detection.
Scott Keck-Warren
8:55
Yeah, and I think it's really kind of fascinating and like the 2 kind of pieces that I always like to run these tools. So this is like a static code analysis tool. Which, which I've talked about before, like in the past. So you guys can find like other articles and videos and stuff about that. But like, I like to run it like in GitHub Actions and our CI environment. And then also like in a pre-commit situation where like it's checking files before it's leaving, before the commit can be created. And like JSCPD really works well, like in this, both of those environments. Like you, you have like from, was that like an, uh, a, um, like a conscious decision or did it just kind of work out that it works both well in both?
B
Speaker B
9:33
Say, for now, one more place where GSCPD works. Every time when you run agentic coding, agentic loop, you can attach skills or MCP server and run copy-paste detector with agentic coding and get feedback before commit before any other actions, before continuous integration. It's useful as well. And I can see that based on statistic downloads and use cases, most of questions related to this agentic approach. And so I would say the new era of JSCPD started probably one year ago.
Scott Keck-Warren
10:26
Yeah, I, I think that that's been like a, a good, like piece to, to focus on too with this is that like for, especially in like the projects that I've worked on that are, are agentically driven, um, like it just, it will, it doesn't care about like how many times it like rewrites the same code. It will, it'll just like regenerate the same code over and over again. And it's not necessarily copying and pasting cuz it's like regenerating it from scratch. But then like you have that like additional overhead of maintenance, like whenever you have to change one of those things. Um, so I think like, yes, like I've thrown this also in like my, one of my tools as one of my tools for my agents to run so that then they go, oh no, I should fix this. This is, this would be a better like solution essentially. So yeah, so it's really good. We'll be right back after this word from our partners. As a programmer, you know that code reviews are critical, but are also time consuming. CodeRabbit acts as your AI copilot, providing instant code review comments and potential impacts of every pull request. Beyond just flagging issues, CodeRabbit provides one-click fix solutions and lets you define custom code quality rules using AST grep patterns, catching subtle issues that traditional static analysis tools might miss. CodeRabbit reviews 1 million pull requests every week across 3 million repositories and is used by 100,000 open source projects. CodeRabbit is free for all open source project repos. Get started today at phpa.me/coderabbit. That's phpa.me/coderabbit. So part of it is like a skill, right? And so like, how does the skill work? Um, is it, is it like calling the JCP— uh, JSCPD?
B
Speaker B
12:02
JSCPD, JavaScript Copy Paste Data. It was originally on JavaScript or CoffeeScript. So, uh, yeah, uh, skill, uh, couple of skills shipped together with copy-paste detector, and it's like regular skills, markdown with explanation how to work with copy-paste detector. First one is GSCPD, just list of available options and ways to run this tool. And second one, it's DRY, don't repeat yourself refactoring skill. And there I— Yeah. Suggest to find the most important pieces of code what copied from one place to another place and suggest to change this code in proper way. So reorganize existing one code and rerun copy-paste detector and make code more clear and understandable and remove duplicates.
Scott Keck-Warren
13:13
It's really slick.
B
Speaker B
13:14
Yeah, yeah.
Scott Keck-Warren
13:17
Yeah, it's— I, sorry, go ahead.
B
Speaker B
13:21
It's a really small description and I guess it can help people to keep the code clear. I'm going to extend it, probably add some information about statistic or about codebase, something like biggest files or more complex files and attach it to copy-paste report and it can help to make much more deeper refactoring with this knowledge. And also I can say that tokens, it's like new currency for developers and I try to care about tokens as well. And I changed regular copy-paste detection report to optimized, AI optimized report. I remove everything what not needed to agent, just one file, second file, and range of tokens, range of lines where copy-paste detected. And probably It's less, 80% less reports than regular report, and agent can read it, would fix, and don't spend a lot of tokens as well. Like, okay.
Scott Keck-Warren
14:49
We'll put links to that in the description. Yeah, we'll put links to some of that stuff in the description. And then, so before we do let you go today, is there anything that you'd like to direct our listeners to? Like, do you have like the, where can people find those articles, I guess is the best question.
B
Speaker B
15:08
Didn't get the question.
Scott Keck-Warren
15:10
Wait, so like where? Yeah, so where can people find those articles? Like what's the— are they on the JavaScript copy paste detector website?
B
Speaker B
15:21
Provide links. I have a couple of articles on HackerNoon platform. Also, I have article on Medium and But the main hub, it's jscpd.dev, and I try to keep updated documentation, and sometimes I put some links in the documentation.
Scott Keck-Warren
15:45
Okay, that sounds awesome. And so thank you so much for your time today. I really appreciate both the project and your time today to talk to us.
B
Speaker B
15:53
Yes, thank you for the invite. And I wish you a good audience for your channel and so interesting technical topics. I subscribed to the channel as well.
Scott Keck-Warren
16:13
Well, that's good to hear. Yeah. So that's the end of the interview. I do really appreciate like both your time and the project. I'm gonna let you go. I have to record like the intro and the outro, and I'm not gonna make you sit through all that 'cause it's really quite boring. So thank you. I do appreciate everything. And I, and I think like probably in addition to this will probably come out in August, like the interview, and there probably will be a companion piece in the, in our magazine as well to kind of talk about this. I try to like keep those, I try to like do, we try to do those as well. So there might be a companion piece for this and maybe an educational video. We'll have to see.
B
Speaker B
16:51
By the way, if you have some, like, not printed version, but you have a text version of your magazine, I can try to write some article as well. I try to be a writer, and that's why I suggest my investigation and my articles.
Scott Keck-Warren
17:12
I will forward your contact information to the editors. and tell them that you're interested in doing that because they're the ones that handle like more than what I— I just give them an article to write or that I wrote. They handle like all of the special features and stuff like that. But I will be more than happy to pass you along if you're okay with that.
B
Speaker B
17:31
Okay.
Scott Keck-Warren
17:32
Okay. Awesome. So thank you again. It was nice meeting you.
B
Speaker B
17:36
Thank you. Nice to meet you as well. Have a great day.
Scott Keck-Warren
17:41
I have to say another heartfelt thank you to Andre for all of his time as well as for you for listening. If you're not already a subscriber, make sure you subscribe with whatever your preferred method is. This video, this episode is available. This episode is available both in an audio version on our website, as well as a video version on our YouTube channel. If you learned something about JavaScript copy paste detector, please share this episode with a friend or a coworker, as it does help others find us. This is Scott Keck Warren for the PHP Architect Community Corner signing off and reminding you to keep listening, keep coding, and keep reading.