How "find custom html text in urls"?

juanchi · Mar 15, 2021

Hi sorry if this is not the correct thread, I have an important question:

Do you know any software that can "find custom html text in urls"? For example I have 20 URLs and I need to filter only those that have a specific word.

MisterXYZ · Mar 15, 2021

You can do it in excel or any other online tool like google sheets.

juanchi · Mar 15, 2021

MisterXYZ said:
You can do it in excel or any other online tool like google sheets.

thanks, but in an excel ??

Examples, I just found 3 URLs

Code:

https://www.goodhousekeeping.com/appliances/washer-reviews/g105/washing-machine-reviews/
https://www.thespruce.com/best-washing-machines-4107080
https://www.tomsguide.com/us/best-washing-machines,review-6208.html

And I only want to extract the URLs that the text contains in the html:
"washing machines in 2021"

If we do the filter we only obtain the URL:

Code:

https://www.tomsguide.com/us/best-washing-machines,review-6208.html

That's what he was talking about.

MisterXYZ · Mar 15, 2021

In that case, you have to download url pages and then you can use windows search or linux command to search.

juanchi · Mar 16, 2021

loopline
Hello!

I quote you to ask if scrapebox has any functions/options or addons to do this job?

SilverLine22 · Mar 16, 2021

You can try a trick, Download phpDesigner8 IDE from http://www.mpsoftware.dk/downloads.php with 30-days free trial with email only.

After installation, open the file into it and use its search & replace function even in nested files directory as well.

Make sure to make your phrase like Washing Machine in 2021 to washing-machine-in-2021 and just replace with your own words.

Hope this helps.

chseoservice · Mar 16, 2021

how you retrieve those urls? where you store them. Than we might can provide a solution to your problem

juanchi · Mar 16, 2021

AllenTom said:
You can try a trick, Download phpDesigner8 IDE from http://www.mpsoftware.dk/downloads.php with 30-days free trial with email only.

After installation, open the file into it and use its search & replace function even in nested files directory as well.

Make sure to make your phrase like Washing Machine in 2021 to washing-machine-in-2021 and just replace with your own words.

Hope this helps.

The software seems quite functional and powerful for creating websites

I just ran a test with the steps indicated, and I get these results:

Uhmmm this is the configuration?

SilverLine22 · Mar 17, 2021

you forgot following step:

"Make sure to make your phrase like Washing Machine in 2021 to washing-machine-in-2021 and just replace with your own words."

chseoservice · Mar 17, 2021

I would use a simple java program:

Code:

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;

public class SimpleWebScraper {

    public static void main(String[] args) throws Exception {

        SimpleWebScraper ws = new SimpleWebScraper();

        String keyPhrase = "washing machines in 2021";

        String[] urls = new String[]{

                "https://www.goodhousekeeping.com/appliances/washer-reviews/g105/washing-machine-reviews/",
                "https://www.thespruce.com/best-washing-machines-4107080",
                "https://www.tomsguide.com/us/best-washing-machines,review-6208.html"
        };

        for (String url : urls) {
            URL web;
            InputStream is = null;
            BufferedReader br;
            String line;

            web = new URL(url);
            is = web.openStream();  // throws an IOException
            br = new BufferedReader(new InputStreamReader(is));

            while ((line = br.readLine()) != null) {
                //System.out.println(line);

                if (line.contains(keyPhrase)) {
                    System.out.println("key phrase found in: " + url);
                    System.out.println(line);
                }
            }
            is.close();
        }

    }
}

output:

key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<title>Best washing machines in 2021 | Tom's Guide</title>
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<meta name="parsely-title" content="Best washing machines in 2021">
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
"name": "Best washing machines in 2021",
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
"headline": "Best washing machines in 2021",
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<h1>Best washing machines in 2021</h1>

juanchi · Mar 17, 2021

chseoservice said:
I would use a simple java program:

Code:

import java.io.BufferedReader; import java.io.InputStream; import java.io.InputStreamReader; import java.net.URL; public class SimpleWebScraper { public static void main(String[] args) throws Exception { SimpleWebScraper ws = new SimpleWebScraper(); String keyPhrase = "washing machines in 2021"; String[] urls = new String[]{ "https://www.goodhousekeeping.com/appliances/washer-reviews/g105/washing-machine-reviews/", "https://www.thespruce.com/best-washing-machines-4107080", "https://www.tomsguide.com/us/best-washing-machines,review-6208.html" }; for (String url : urls) { URL web; InputStream is = null; BufferedReader br; String line; web = new URL(url); is = web.openStream(); // throws an IOException br = new BufferedReader(new InputStreamReader(is)); while ((line = br.readLine()) != null) { //System.out.println(line); if (line.contains(keyPhrase)) { System.out.println("key phrase found in: " + url); System.out.println(line); } } is.close(); } } }

output:

key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<title>Best washing machines in 2021 | Tom's Guide</title>
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<meta name="parsely-title" content="Best washing machines in 2021">
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
"name": "Best washing machines in 2021",
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
"headline": "Best washing machines in 2021",
key phrase found in: https://www.tomsguide.com/us/best-washing-machines,review-6208.html
<h1>Best washing machines in 2021</h1>

Hello thanks for sharing the code, but could you show in an image how you got the results?

I am not a java programmer

juanchi · Mar 17, 2021

AllenTom said:
you forgot following step:

"Make sure to make your phrase like Washing Machine in 2021 to washing-machine-in-2021 and just replace with your own words."

I did the tests with the 2 texts, but still the results were not obtained.

chseoservice · Mar 17, 2021

juanchi said:
could you show in an image how you got the results?

you need a java (java script is not java) ide for that like:

https://www.jetbrains.com/idea/download/

chseoservice · Mar 17, 2021

Code:

<?php
    $arr = array( "https://www.goodhousekeeping.com/appliances/washer-reviews/g105/washing-machine-reviews/", "https://www.thespruce.com/best-washing-machines-4107080", "https://www.tomsguide.com/us/best-washing-machines,review-6208.html" );
    foreach ($arr as &$value) {
      $text = file_get_contents($value);
      $intext = strpos($text, 'washing machines in 2021');
      if ( $intext !== false)
        echo 'found in ' . $value;
    }
?>

php version:

oiviivio · Mar 17, 2021

chseoservice said:

Code:

<?php
    $arr = array( "https://www.goodhousekeeping.com/appliances/washer-reviews/g105/washing-machine-reviews/", "https://www.thespruce.com/best-washing-machines-4107080", "https://www.tomsguide.com/us/best-washing-machines,review-6208.html" );
    foreach ($arr as &$value) {
      $text = file_get_contents($value);
      $intext = strpos($text, 'washing machines in 2021');
      if ( $intext !== false)
        echo 'found in ' . $value;
    }
?>

php version:

Nice. I think .net is better with a lot of urls

harrison21 · Mar 18, 2021

http://findandreplace.io/

oiviivio · Mar 18, 2021

harrison21 said:
http://findandreplace.io/

Find in website not in files

How "find custom html text in urls"?

juanchi

Registered Member

MisterXYZ

Elite Member

juanchi

Registered Member

MisterXYZ

Elite Member

juanchi

Registered Member

SilverLine22

Power Member

chseoservice

Registered Member

juanchi

Registered Member

SilverLine22

Power Member

chseoservice

Registered Member

juanchi

Registered Member

juanchi

Registered Member

chseoservice

Registered Member

chseoservice

Registered Member

oiviivio

Newbie

harrison21

Newbie

oiviivio

Newbie

Main Menu

Marketplace

Making Money

BlackHat World