Facing slowness while running JSCPD

[Sudharshan45]

Hi kucherenko,
I am having 55,000 lines file in my repo. if we run JSCPD for that file takes more than 10 minutes. Most of the time consumes by Token generation Method name for reference (this.tokenizer.generateMaps(id, text, format, this.options); ) could please suggest solution for reduce time. Thanks in advance :)

[kucherenko]

Hi @Sudharshan45
Sometimes tokenization process is slow, and I have no clear answer on how to fix it, usually, it requires more investigation of the sources.

[benjaminaudet]

it would be very interesting to have a bit of your hardware config @Sudharshan45 just to fast check if it's not hardware related.
Personally I didn't faced any issues running it on a 110k lines of codes. It was a bit long but clearly far away from 10min.
My PC is a Latitude 5520. 16gigs i5

Oh, and out of curiosity, you're not on a hybrid development environment using Windows with WSL2?
If it's the case, assure using jscpd in the same filesystem as where you're repo is located.

[KineticTheory]

I'm having a similar pain-point. jscpd is taking about 15 minutes to parse through 100k LOC. I'm running in GitHub's super-linter image.

[Sudharshan45]

Hi @benjaminaudet, I am using MAC M1 16gb and running JSCPD for single js file containing 55k lines of code. I fixed that issue by slicing single file into multiple files

[benjaminaudet]

Hi @benjaminaudet, I am using MAC M1 16gb and running JSCPD for single js file containing 55k lines of code. I fixed that issue by slicing single file into multiple files

Just realize you were doing it on one file of 55k LOC... I mean, seems logic to me, your PC was bottlenecking at some point.
Next time, make it a bit clearer that you're environment is so particular.
Happy you found a solution, best wishes to you. Issue resolved.