How to scrape specific URLs from the website with scrapbox or screaming frog

domiic3284 · Jan 29, 2025

Hi guys I want to sccrape the specific URLs from the website

Code:

https://ww23.gogoanimes.fi/

In this website I want to scrape only URLs which cantain the word " Epiosde "

Example

Code:

https://ww23.gogoanimes.fi/puniru-wa-kawaii-slime-episode-8

all the types of URLs which cantain the word epiosde I only want these urls not any catagory or etc

I am not good in regix so maybe the scrapbox custom data scraper helps me but can anyne know how can i config screaming frog or scrapbox for this kind of task?

EDIT: this website dont have a sitemap or also sitemap adress is not mention in robot.txt

maresen · Jan 29, 2025

domiic3284 · Jan 29, 2025

maresen said:
here are the urls (no clue how to config screaming frog or scrapbox for this kind of task)

"https://ww19.gogoanimes.fi/copihan-episode-1",
"https://ww23.gogoanimes.fi/copihan-episode-2",
"https://ww23.gogoanimes.fi/copihan-episode-3",
"https://ww23.gogoanimes.fi/copihan-episode-4",
"https://ww23.gogoanimes.fi/copihan-episode-5",
"https://ww23.gogoanimes.fi/copihan-episode-6",
"https://ww23.gogoanimes.fi/copihan-episode-7",
"https://ww9.gogoanimes.fi/major-episode-42",
"https://ww23.gogoanimes.fi/major-episode-41",
"https://ww23.gogoanimes.fi/major-episode-43",
"https://ww23.gogoanimes.fi/major-episode-1",
"https://ww23.gogoanimes.fi/major-episode-2",
"https://ww23.gogoanimes.fi/major-episode-3",
"https://ww23.gogoanimes.fi/major-episode-4",
"https://ww23.gogoanimes.fi/major-episode-5",
"https://ww23.gogoanimes.fi/major-episode-6",
"https://ww23.gogoanimes.fi/major-episode-7",
"https://ww23.gogoanimes.fi/major-episode-8",
"https://ww23.gogoanimes.fi/major-episode-9",
"https://ww23.gogoanimes.fi/major-episode-10",
"https://ww23.gogoanimes.fi/major-episode-11",
"https://ww23.gogoanimes.fi/major-episode-12",
"https://ww23.gogoanimes.fi/major-episode-13",
"https://ww23.gogoanimes.fi/major-episode-14",
"https://ww23.gogoanimes.fi/major-episode-15",
"https://ww23.gogoanimes.fi/major-episode-16",
"https://ww23.gogoanimes.fi/major-episode-17",
"https://ww23.gogoanimes.fi/major-episode-18",
"https://ww23.gogoanimes.fi/major-episode-19",
"https://ww23.gogoanimes.fi/major-episode-20",
"https://ww23.gogoanimes.fi/major-episode-21",
"https://ww23.gogoanimes.fi/major-episode-22",
"https://ww23.gogoanimes.fi/major-episode-23",
"https://ww23.gogoanimes.fi/major-episode-24",
"https://ww23.gogoanimes.fi/major-episode-25",
"https://ww23.gogoanimes.fi/major-episode-26",
"https://ww23.gogoanimes.fi/major-episode-27",
"https://ww23.gogoanimes.fi/major-episode-28",
"https://ww23.gogoanimes.fi/major-episode-29",
"https://ww23.gogoanimes.fi/major-episode-30",
"https://ww23.gogoanimes.fi/major-episode-31",
"https://ww23.gogoanimes.fi/major-episode-32",
"https://ww23.gogoanimes.fi/major-episode-33",
"https://ww23.gogoanimes.fi/major-episode-34",
"https://ww23.gogoanimes.fi/major-episode-35",
"https://ww23.gogoanimes.fi/major-episode-36",
"https://ww23.gogoanimes.fi/major-episode-37",
"https://ww23.gogoanimes.fi/major-episode-38",
"https://ww23.gogoanimes.fi/major-episode-39",
"https://ww23.gogoanimes.fi/major-episode-40",
"https://ww23.gogoanimes.fi/major-episode-42",
"https://ww23.gogoanimes.fi/major-episode-44",
"https://ww23.gogoanimes.fi/major-episode-45",
"https://ww23.gogoanimes.fi/major-episode-46",
"https://ww23.gogoanimes.fi/major-episode-47",
"https://ww23.gogoanimes.fi/major-episode-48",
"https://ww23.gogoanimes.fi/major-episode-49",
"https://ww23.gogoanimes.fi/major-episode-50",
"https://ww23.gogoanimes.fi/major-episode-51",
"https://ww23.gogoanimes.fi/major-episode-52",
"https://ww23.gogoanimes.fi/major-episode-53",
"https://ww23.gogoanimes.fi/major-episode-54",
"https://ww23.gogoanimes.fi/major-episode-55",
"https://ww23.gogoanimes.fi/major-episode-56",
"https://ww23.gogoanimes.fi/major-episode-57",
"https://ww23.gogoanimes.fi/major-episode-58",
"https://ww23.gogoanimes.fi/major-episode-59",
"https://ww23.gogoanimes.fi/major-episode-60",
"https://ww23.gogoanimes.fi/major-episode-61",
"https://ww23.gogoanimes.fi/major-episode-62",
"https://ww23.gogoanimes.fi/major-episode-63",
"https://ww23.gogoanimes.fi/major-episode-64",
"https://ww23.gogoanimes.fi/major-episode-65",
"https://ww23.gogoanimes.fi/major-episode-66",
"https://ww23.gogoanimes.fi/major-episode-67",
"https://ww23.gogoanimes.fi/major-episode-68",
"https://ww23.gogoanimes.fi/major-episode-69",
"https://ww23.gogoanimes.fi/major-episode-70",
"https://ww23.gogoanimes.fi/major-episode-71",
"https://ww23.gogoanimes.fi/major-episode-72",
"https://ww23.gogoanimes.fi/major-episode-73",
"https://ww23.gogoanimes.fi/major-episode-74",
"https://ww23.gogoanimes.fi/major-episode-75",
"https://ww23.gogoanimes.fi/major-episode-76",
"https://ww23.gogoanimes.fi/major-episode-77",
"https://ww23.gogoanimes.fi/major-episode-78",
"https://ww23.gogoanimes.fi/major-episode-79",
"https://ww23.gogoanimes.fi/major-episode-80",
"https://ww23.gogoanimes.fi/major-episode-81",
"https://ww23.gogoanimes.fi/major-episode-82",
"https://ww23.gogoanimes.fi/major-episode-83",
"https://ww23.gogoanimes.fi/major-episode-84",
"https://ww23.gogoanimes.fi/major-episode-85",
"https://ww23.gogoanimes.fi/major-episode-86",
"https://ww23.gogoanimes.fi/major-episode-87",
"https://ww23.gogoanimes.fi/major-episode-88",
"https://ww23.gogoanimes.fi/major-episode-89",
"https://ww23.gogoanimes.fi/major-episode-90",
"https://ww23.gogoanimes.fi/major-episode-91",
"https://ww23.gogoanimes.fi/major-episode-92",
"https://ww23.gogoanimes.fi/major-episode-93",
"https://ww23.gogoanimes.fi/major-episode-94",
"https://ww23.gogoanimes.fi/major-episode-95",
"https://ww23.gogoanimes.fi/major-episode-96",
"https://ww23.gogoanimes.fi/major-episode-97",
"https://ww23.gogoanimes.fi/major-episode-98",
"https://ww23.gogoanimes.fi/major-episode-99",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-9",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-8",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-7",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-6",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-5",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-4",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-3",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-2",
"https://ww23.gogoanimes.fi/nageki-no-bourei-wa-intai-shitai-episode-1",
"https://ww23.gogoanimes.fi/idinvaded-episode-1",
"https://ww23.gogoanimes.fi/idinvaded-episode-2",
"https://ww23.gogoanimes.fi/idinvaded-episode-3",
"https://ww23.gogoanimes.fi/idinvaded-episode-4",
"https://ww23.gogoanimes.fi/idinvaded-episode-5",
"https://ww23.gogoanimes.fi/idinvaded-episode-6",
"https://ww23.gogoanimes.fi/idinvaded-episode-7",
"https://ww23.gogoanimes.fi/idinvaded-episode-8",
"https://ww23.gogoanimes.fi/idinvaded-episode-9",
"https://ww23.gogoanimes.fi/idinvaded-episode-10",
"https://ww23.gogoanimes.fi/idinvaded-episode-11",
"https://ww23.gogoanimes.fi/idinvaded-episode-12",
"https://ww23.gogoanimes.fi/idinvaded-episode-13",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-7",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-6",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-5",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-4",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-3",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-2",
"https://ww23.gogoanimes.fi/rezero-kara-hajimeru-isekai-seikatsu-3rd-season-episode-1",
"https://ww23.gogoanimes.fi/another-anime-episode-1",
"https://ww23.gogoanimes.fi/another-anime-episode-2",
"https://ww23.gogoanimes.fi/another-anime-episode-3",
"https://ww23.gogoanimes.fi/another-anime-episode-4",
"https://ww23.gogoanimes.fi/another-anime-episode-5",
"https://ww23.gogoanimes.fi/another-anime-episode-6",
"https://ww23.gogoanimes.fi/another-anime-episode-7",
"https://ww23.gogoanimes.fi/another-anime-episode-8",
"https://ww23.gogoanimes.fi/another-anime-episode-9",
"https://ww23.gogoanimes.fi/another-anime-episode-10",
"https://ww23.gogoanimes.fi/another-anime-episode-11",
"https://ww23.gogoanimes.fi/another-anime-episode-12",
"https://ww23.gogoanimes.fi/dandadan-episode-1",
"https://ww23.gogoanimes.fi/dandadan-episode-2",
"https://ww23.gogoanimes.fi/dandadan-episode-3",
"https://ww23.gogoanimes.fi/dandadan-episode-4",
"https://ww23.gogoanimes.fi/dandadan-episode-5",
"https://ww23.gogoanimes.fi/dandadan-episode-6",
"https://ww23.gogoanimes.fi/dandadan-episode-7",
"https://ww23.gogoanimes.fi/dandadan-episode-8",
"https://ww23.gogoanimes.fi/re-main-episode-1",
"https://ww23.gogoanimes.fi/re-main-episode-2",
"https://ww23.gogoanimes.fi/re-main-episode-3",
"https://ww23.gogoanimes.fi/re-main-episode-4",
"https://ww23.gogoanimes.fi/re-main-episode-5",
"https://ww23.gogoanimes.fi/re-main-episode-6",
"https://ww23.gogoanimes.fi/re-main-episode-7",
"https://ww23.gogoanimes.fi/re-main-episode-8",
"https://ww23.gogoanimes.fi/re-main-episode-9",
"https://ww23.gogoanimes.fi/re-main-episode-10",
"https://ww23.gogoanimes.fi/re-main-episode-11",
"https://ww23.gogoanimes.fi/re-main-episode-12",
"https://ww23.gogoanimes.fi/free--episode-1",
"https://ww23.gogoanimes.fi/free--episode-2",
"https://ww23.gogoanimes.fi/free--episode-3",
"https://ww23.gogoanimes.fi/free--episode-4",
"https://ww23.gogoanimes.fi/free--episode-5",
"https://ww23.gogoanimes.fi/free--episode-6",
"https://ww23.gogoanimes.fi/free--episode-7",
"https://ww23.gogoanimes.fi/free--episode-8",
"https://ww23.gogoanimes.fi/free--episode-9",
"https://ww23.gogoanimes.fi/free--episode-10",
"https://ww23.gogoanimes.fi/free--episode-11",
"https://ww23.gogoanimes.fi/free--episode-12"

How you have scraped all these??

CPANathan · Jan 29, 2025

domiic3284 said:
How you have scraped all these??

Scrapebox I assume

It's actually not that hard to scrape lesser-developed sites; ask ChatGPT for a plan for a Python script then ask Claude for a hand piecing it together

hideath · Jan 29, 2025

Make a Python script that will go through the anime listing pages.
Should start with "https://ww23.gogoanimes.fi/anime-list.html?page=1", incrementing the page number until it returns 404,
Scrape and append the URLs/names from the li HTML elements located in the div with class "anime_list_body" to a stylesheet/txt file,
It should be pretty easy to do, just a Python script using BS4 (or Selenium if needed), which could be done with a couple of prompts

maresen · Jan 29, 2025

domiic3284 said:
How you have scraped all these??

https://www.firecrawl.dev/extract

How to scrape specific URLs from the website with scrapbox or screaming frog

domiic3284

Regular Member

maresen

Junior Member

domiic3284

Regular Member

CPANathan

Elite Member

hideath

Elite Member

maresen

Junior Member

Main Menu

Marketplace

Making Money

BlackHat World