Text Editor to Search for and Delete Lines in Keyword Lists?

[Alice Mizer]

I've downloaded something like 20 text editors tonight trying to find one to suit my needs. Basically, I need a text editor capable of handling extremely large keyword/URL lists--which is easy in and of itself.

However, I really want to find a text editor that will search for a particular string and delete any line containing that string for filtering purposes. I know I can use the basic regular expression search in a lot of advanced text editors, but it's not as friendly as a text editor that does pattern matching and you can only search for one thing at a time.

I found a program that loads a list from a file, compares against a list of words, and deletes all the lines containing those words--and it doesn't work on my computer, haha! Go figure.

If anyone has something like this that they'd like to share their knowledge on, I'd love to hear about it!

 

[Standard Toaster]

I've downloaded something like 20 text editors tonight trying to find one to suit my needs. Basically, I need a text editor capable of handling extremely large keyword/URL lists--which is easy in and of itself.

However, I really want to find a text editor that will search for a particular string and delete any line containing that string for filtering purposes. I know I can use the basic regular expression search in a lot of advanced text editors, but it's not as friendly as a text editor that does pattern matching and you can only search for one thing at a time.

I found a program that loads a list from a file, compares against a list of words, and deletes all the lines containing those words--and it doesn't work on my computer, haha! Go figure.

If anyone has something like this that they'd like to share their knowledge on, I'd love to hear about it!

I can easily create something that does this with win automation. Feel free to hit me up via PM with full details, I'd do it for free for you. Or you can try and get the grip of this program on your own.

Good luck.

 

[Standard Toaster]

I found a program that loads a list from a file, compares against a list of words, and deletes all the lines containing those words--and it doesn't work on my computer, haha! Go figure.

I made the attached program based on this description. The exe is enough to run the program. The .waj is the source code for the win automation console if you want to have a look.

The program works on regex. So, if you enter the keyword as "possible", it will match "impossible". To overcome this, you can use keyword " possible" (with a space at the beginning).

You cannot directly enter any regex as a keyword!

Hope that helps.

VirusTotals:

http://www.virustotal.com/file-scan/report.html?id=ecc95ce32dae13272fe9ac5f08ad4f28fdd5c43d77bdad40f30ef0e5568c783d-1286556936
http://www.virustotal.com/file-scan/report.html?id=e4d4823fd9318b3ecc40e37d2cf81f97a09a93402021965723c3309e6b623c2e-1286557045

 

[kveldulv]

Try textpipe. THe swiss army chainsaw of text
h()p://vv07*softarchive*net/portable_textpipe_pro_ml.147339.html

Otherwise..., EmEditor , or upload to a unix box and brush up on your grep/awk

 

[jazzc]

In notepad++:

Go to the "Replace" command (Press Ctrl+H).

Find what: Your regular expression here that will match the whole line[*]
Replace with: leave this empty
Press the replace all button.
Done!

[*]
Suppose our blacklisted string is "WhiteHat"
The regular expression to match WhiteHat is well... WhiteHat.
To make it select the whole line, the new regular expression would be:

(.*)WhiteHat(.*)

 

[Alice Mizer]

Try textpipe. THe swiss army chainsaw of text
h()p://vv07*softarchive*net/portable_textpipe_pro_ml.147339.html

Otherwise..., EmEditor , or upload to a unix box and brush up on your grep/awk

The program that Toaster wrote for me is very lightweight and works great, but I also tried this program out as I've also been looking for that "swiss army chainsaw" of text editors. This is a really great find, too--it lets me remove duplicate lines, sort my lists, and use a separate file to filter out unwanted keywords/URLs and without freezing up and crapping out like other programs. Basically, everything I needed in one package.

Thanks for the advice everyone! c:

 

[Alex881]

In notepad++:

Go to the "Replace" command (Press Ctrl+H).

Find what: Your regular expression here that will match the whole line
[*]
Replace with: leave this empty
Press the replace all button.
Done!
[*]
Suppose our blacklisted string is "WhiteHat"
The regular expression to match WhiteHat is well... WhiteHat.
To make it select the whole line, the new regular expression would be:

(.*)WhiteHat(.*)

+1 for Notepad ++. It's works very fast and doesn't choke your processor even with huge lists.

 

[Alice Mizer]

+1 for Notepad ++. It's works very fast and doesn't choke your processor even with huge lists.

Haha, I really like Notepad++ but I don't think there's a way you can go through an entire "blacklist" and delete any line in another file containing those terms completely automated? I tried messing around with regexps but couldn't do anymore than a basic search and replace operation, but it is a really great piece of software regardless.

 

[apekillape]

Haha, I really like Notepad++ but I don't think there's a way you can go through an entire "blacklist" and delete any line in another file containing those terms completely automated? I tried messing around with regexps but couldn't do anymore than a basic search and replace operation, but it is a really great piece of software regardless.

That was gonna be my suggestion too. If there are too many different items to manually ctrl+replace though, I'd probably just script up something in php/mysql to check fields against another table and kill them if they match.

If you don't know any php or anything, I could probably write something up like that for you if you want. It's pretty quick and I ain't busy.

N

 

[lvsniper]

my favorites would be notepad++ & vim.

 

[CheapAlert]

Uh original Notepad can do this?

 

[s4nt0s]

I use textcrawler to do stuff like this. http://www.digitalvolcano.co.uk/content/textcrawler

 

[smartalex4]

Can anyone tell me how I can use one of these programs to quickly remove all webpages that end in .doc, .pdf, .txt, and .flv files from a list that I scrapped?

Thanks in advance

 

[kieran84]

best software is MS Word

 

[kieran84]

i am using microsoft word 2010. this is a good software. try it. u will find ur answer

 

[smartalex4]

Well I figured out how to use Text Crawler.

For anyone else that has the same problem use this:
.*pdf.*(\r|\n|\r\n)

Where pdf is the extension you want to remover....and it will remove the entire line.

 

[harry]

"Batch keyword cleaner" is great for this sort of thing.

 

[dog-tag]

I just use notepad 2 for things like that, its free! Maybe its not powerful enough for you but I do similar things like that with it with no problems.

I got some pirate versions of text editors but they all were found out over time and became useless. Take a look at "notepad 2" anyway..

 

[sfidirectory]

I'm using TED Notepad For Windows for this task, although I could stop being lazy and make a Java program to do this instead. I'm thinking this Java program will have a JTextField where you enter the particular string, a JButton that opens the .txt based file in the same JPanel (in a JTextArea or something like that), and when you press a button (that could say something like "go"), a looping mechanism will go through each line in the .txt file and delete the lines containing the string you entered in the JTextField. It will depend if your txt has ASCII characters or not. Then when the program has finished deleting the particular lines, there will be a "save" button where you can save the new file without the lines you filtered out using the string entered in the JTextField.

Sorry for blabbering on, am trying to think like a programmer until my next semester starts in a couple months .

 

[xpressioniz]

I'm using keyword pad from s0ftnik, great for filtering huge list of keywords