What’s your discord?Yeah mate reach out to me either on tg or disc
I am more active on disc tho
What’s your discord?Yeah mate reach out to me either on tg or disc
I am more active on disc tho
@igetcuzWhat’s your discord?
Added you I’m fgsxk@igetcuz
ok, i've tried this out and doesn't seem to be saving the walle.. and tried using chatgpt to fix it, create the wallet file.. but sill no address..I made a script to automate this. You can provide a list of dexscreener urls and it will go through each one, scrape the top 100 wallets, and save them to a wallets.json frequency map.
https://pastebin.com/QTSRFsHy
How to run it:
1. Install Node.js from nodejs.org (LTS version)
2. Make a new folder (e.g. wallet-scraper) and navigate to it.
3. Create a file called index.js and paste the code from pastebin to it.
4. Open a terminal (Command Prompt on Windows) in the folder you created and initialize the project:
Code:npm init es6 -y
5. Install packages:
Code:npm install puppeteer-extra puppeteer-extra-plugin-stealth
6. Add your own list of URLs by editing the const urls array in index.js, or use a custom file:
- Create a file called urls.json
- Put your URLs in this format:
Code:[ "https://dexscreener.com/solana/url1", "https://dexscreener.com/solana/url2" ]
7. Run the script:
Code:node index.js # If you added your URLs to index.js node index.js --urls urls.json # If you are using a custom file
You can also choose to add more wallets to the existing wallets.json frequency map by using the --no-overwrite argument:
Code:node index.js --no-overwrite
You can also use --help argument if you need help:
Code:node index.js --help
Please note I haven't tested it extensively so errors might occur. Also be aware that the script might stop working if DexScreener decide to change their site.
If you see a Cloudflare check you have to solve it manually.
Try not to scrape too many URLs at a time so you don't lose progress. I tested up to 20 and there was no problem but you never know. You can always use --no-overwrite to continue with the same frequency map.
Always back up your wallets.json file by renaming it if you don't want to lose it.
Good luck!
The pastebin link does not work anymoreI made a script to automate this. You can provide a list of dexscreener urls and it will go through each one, scrape the top 100 wallets, and save them to a wallets.json frequency map.
https://pastebin.com/QTSRFsHy
How to run it:
1. Install Node.js from nodejs.org (LTS version)
2. Make a new folder (e.g. wallet-scraper) and navigate to it.
3. Create a file called index.js and paste the code from pastebin to it.
4. Open a terminal (Command Prompt on Windows) in the folder you created and initialize the project:
Code:npm init es6 -y
5. Install packages:
Code:npm install puppeteer-extra puppeteer-extra-plugin-stealth
6. Add your own list of URLs by editing the const urls array in index.js, or use a custom file:
- Create a file called urls.json
- Put your URLs in this format:
Code:[ "https://dexscreener.com/solana/url1", "https://dexscreener.com/solana/url2" ]
7. Run the script:
Code:node index.js # If you added your URLs to index.js node index.js --urls urls.json # If you are using a custom file
You can also choose to add more wallets to the existing wallets.json frequency map by using the --no-overwrite argument:
Code:node index.js --no-overwrite
You can also use --help argument if you need help:
Code:node index.js --help
Please note I haven't tested it extensively so errors might occur. Also be aware that the script might stop working if DexScreener decide to change their site.
If you see a Cloudflare check you have to solve it manually.
Try not to scrape too many URLs at a time so you don't lose progress. I tested up to 20 and there was no problem but you never know. You can always use --no-overwrite to continue with the same frequency map.
Always back up your wallets.json file by renaming it if you don't want to lose it.
Good luck!
The pastebin link does not work anymore
Can i have a new link?
Just use the V2 script, it works better: https://pastebin.com/4JvWTUFbok, i've tried this out and doesn't seem to be saving the walle.. and tried using chatgpt to fix it, create the wallet file.. but sill no address..
it was working last night but now it doesnt working, im getting this error.To use the script on Windows, you’ll need a code editor such as Notepad++, Visual Studio Code, or Sublime Text. Next, as mentioned in the original post, install Node.js (LTS version) from the official website
Once installed, create a folder in your desktop named (e.g., wallet-scraper). Inside this folder, create a file namedindex.jsusing your code editor (create a new file, set the language to JavaScript, copy/paste the content from the PasteBin link: https://pastebin.com/raw/QTSRFsHy, and save it asindex.jsin the folder you just created).
Copy the file path of the folder from the file explorer’s address bar.
Next, using the Command Prompt, navigate to this folder. For example, if the folder path is
C:\Users\Samsung\Desktop\wallet-scraper:
- Open the Command Prompt (press Windows + R, type cmd, and press Enter).
- In the Command Prompt, type cd followed by the folder path to navigate to the folder. In this example, you would type or copy paste the path e.g.
cd C:\Users\Samsung\Desktop\wallet-scraper- Press Enter.
The Command Prompt will now be in the folder you created.
Install the required dependencies and initialize the project by copying and pasting the following command into the Command Prompt, then press Enter:
npm init es6 -y && npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth fs path url
The folder will now look like this. Ignoreurls.json, as I added it myself
Once done, you will be ready to run the script. The next step is to add the URLs you previously scraped from Dexscreener, as explained in the original post.
To make it easier to scrape all the links for each token on every page of the main Dexscreener table, use a link scraper like the one I provided as an example here:
To add them to theindex.jsfile, locate the line around 308 and add the URLs like this:
JavaScript:// Add the urls you want to scrape here, or create your own urls.json file and provide it as an argument in the CLI. const urls = [ "https://dexscreener.com/solana/xxyxx", "https://dexscreener.com/solana/xxxyx", "https://dexscreener.com/solana/xxxxxyx" ];
Alternatively, you can create a file namedurls.jsonusing your code editor. Paste the URLs in the following format, select JSON as the file type, and save it in the same folder asindex.js:
JSON:[ "https://dexscreener.com/solana/xxyxx", "https://dexscreener.com/solana/xxxyx", "https://dexscreener.com/solana/xxxxxyx" ]
Then, go back to the Command Prompt. If you added the URLs directly intoindex.js, run the script by typing:
node index.js --no-overwrite
If you are using theurls.jsonfile, run the following command instead:
node index.js --urls urls.json --no-overwrite
The script should now start running.
Here is an example using the command:node index.js --urls urls.json --no-overwritewith 2 urls.
The script also creates a file calledwallets.json, where you will find all the scraped wallets.
As highlighted in the original post, avoid analyzing too many URLs at once to prevent triggering Cloudflare's security. Scraping 10-20 URLs at a time with pauses in between should be sufficient.
Good luck!
PS C:\Users\Admin\Desktop\BYBIT\wallet-scrapper> node index.js --urls my-urls.json --no-overwrite --skip-processed
Skipping 0 previously processed URLs
Processing URL 1/1: https://dexscreener.com/solana/ha4xw7d5h5phah43ak24uupr8gbqaam6htoygw1b2wcj
No stored cookies found or error loading cookies
Navigating to page...
Checking for Cloudflare challenge...
Cloudflare challenge detected, waiting for completion...
Error handling Cloudflare challenge: ProtocolError: Protocol error (Runtime.callFunctionOn): Argument should belong to the same JavaScript world as target object
at <instance_members_initializer> (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:97:14)
at new Callback (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:101:16)
at CallbackRegistry.create (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:20:26)
at Connection._rawSend (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:89:26)
at CdpCDPSession.send (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:65:33)
at #evaluate (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:541:50)
at async ExecutionContext.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:334:16)
at async IsolatedWorld.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:104:16)
at async CdpJSHandle.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:154:20)
at async CdpElementHandle.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/api/ElementHandle.js:347:20)
Error processing URL https://dexscreener.com/solana/ha4xw7d5h5phah43ak24uupr8gbqaam6htoygw1b2wcj: ProtocolError: Protocol error (Runtime.callFunctionOn): Argument should belong to the same JavaScript world as target object
at <instance_members_initializer> (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:97:14)
at new Callback (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:101:16)
at CallbackRegistry.create (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:20:26)
at Connection._rawSend (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:89:26)
at CdpCDPSession.send (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:65:33)
at #evaluate (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:541:50)
at async ExecutionContext.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:334:16)
at async IsolatedWorld.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:104:16)
at async CdpJSHandle.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:154:20)
at async CdpElementHandle.evaluateHandle (file:///C:/Users/Admin/Desktop/BYBIT/wallet-scrapper/node_modules/rebrowser-puppeteer-core/lib/esm/puppeteer/api/ElementHandle.js:347:20)
Scraping completed!
Wallet Statistics:
Total unique wallets found: 0
Top 10 wallets by frequency (descending):
Blacklist Statistics:
Bad wallets: 0
Tracked wallets: 0
I have not been able to. get it to work. Started over a few times.It appears I can't edit my comments o.0
Wanted to retract my previous request. NatureBoy's (well, the AI's) suggestion of using Puppeteer's Real Browser did the trick. Thanks.