Gscraper Taking Years To Scrape Targets!!!

keith88 said:
What other options do I have?
Click to expand...

You could always reverse engineer the program yourself.

If you are speaking of scrapers, while I have not reverse engineered ScrapeBox, I have never been given any reason to believe that this occurs in ScrapeBox from unexplained and excessive encrypted traffic.
 
Advertise on BHW
Tosmekop said:
Interesting theory of yours, about GScraper. That code doesn't confirm it.
Click to expand...

Your problem, not mine. I have given one way of verifying it, and another beat me to it - using a packet sniffer to see where the traffic is going.

You might also want to ask why registry keys that allow dotnet tracing are deleted when the program starts.

gscrp2_zps40e2ca93.png


You see, I have nothing to prove to you.
 
W130SN said:
I am not a coder but if you are talking about DESCryptoServiceProvider then I suggest you look it up.

If not could you please explain how you drew your conclusion. I am interested.
Click to expand...

I reverse compiled the program when I noticed excessive unexplained encrypted traffic to this:

gstracert_zps81ec3fc7.png


I have known about it for over a year. I Have suggested that it be stopped. It has not.

If you keep in mind what the program does, it is a fairly good program.
 
JustUs said:
You would be mistaken in some of your assumptions. That particular function is used in a couple of different ways depending on where the function is called from. Called from one place the method encrypts the URL, called from another it encrypts the proxy, called from a third it seeks permission to run. There are two separate keys that are used, and I could show them as well. Also, the program is tied to the computer through a few different hardware aspects.

BUT, and a big one, is that I did not put that piece of code up to debate what it means or how it is used. I put it up to demonstrate that I have, in fact, reverse compiled GScraper and just might have a good idea of what I am speaking about. You can cross test what I am speaking about with Wireshark and see the amount of traffic going to China. By putting that small piece of code up that does not reveal anything critical about the program, I have allowed others to some means to determine if I am speaking about what I know.

The code is not for sale or view unless one can demonstrate a need. A want is not good enough, but it is enough to put Hawke on warning. I have privately taken this to his attention before.

So please everyone, save the nit picking and fire up Wireshark to see where data is going when it is encrypted (208.98.62.34) and then ask yourself why the data would be encrypted?
Click to expand...

No nit picking going on. Just posting that function isn't enough proof. And RE'ing a .NET app is nothing to be proud of, it's a cake. You do not need to be a wizard to run unpacking tools or a decompiler. I'm not trying to attack you. In fact I actually believe you, but you'll have to post actual proof. If you have the keys AND the encrypted data, why don't you just decrypt it and post what's being sent to their server? Ten minutes of work to solve the mystery of Gscraper. :)
 
JustUs said:
Your problem, not mine. I have given one way of verifying it, and another beat me to it - using a packet sniffer to see where the traffic is going.

You might also want to ask why registry keys that allow dotnet tracing are deleted when the program starts.

gscrp2_zps40e2ca93.png


You see, I have nothing to prove to you.
Click to expand...

Disabling JIT by removing the registery keys is nothing bad, many do it. :)
 
hatemachine said:
No nit picking going on. Just posting that function isn't enough proof. And RE'ing a .NET app is nothing to be proud of, it's a cake. You do not need to be a wizard to run unpacking tools or a decompiler. I'm not trying to attack you. In fact I actually believe you, but you'll have to post actual proof. If you have the keys AND the encrypted data, why don't you just decrypt it and post what's being sent to their server? Ten minutes of work to solve the mystery of Gscraper. :)
Click to expand...

Because I am trying to get the problem resolved. The author will know his own work.

I have already told you what the data is.

BTW, it is far more than a few minutes work.
 
Well this thread got incredibly interesting...quickly. Thanks for the heads up JustUs
 
hatemachine said:
Disabling JIT by removing the registery keys is nothing bad, many do it. :)
Click to expand...

Til it creates problems with other programs that are running and being traced, fortunately VS rebuilds those keys. Nor does deleting the registry keys disable JIT. MSIL is based on JIT. Nor does RedGate properly decompile the program.

BTW: The notes are mine as I was working through the program.
 
JustUs said:
Til it creates problems with other programs that are running and being traced, fortunately VS rebuilds those keys. Nor does deleting the registry keys disable JIT. MSIL is based on JIT. Nor does RedGate properly decompile the program.

BTW: The notes are mine as I was working through the program.
Click to expand...

I am not sure why you are talking about MSIL. I was saying that those keys are removed to get rid of the Just In Time debugger which creates annoying windows and can screw with certain apps. Here's an example:

http://social.msdn.microsoft.com/Fo...ting-rid-of-justintime-debugger?forum=vsdebug

See the registry values? They're the same. Some have issues with them, and by removing those values you end up having less problems for your customers.

I am aware that RG can't properly decompile it, as it is a huge project and you'd have to change alot of the UI stuff to make it work. Can you actually compile it? I assume based on the bp's you set, that you can.

JustUs said:
Because I am trying to get the problem resolved. The author will know his own work.

I have already told you what the data is.

BTW, it is far more than a few minutes work.
Click to expand...

I don't see why it would take more than a few minutes? We know what encryption method is used ( DES ) and all you need is the initialization vector and the key which was used to encrypt the data, also you have the encrypted data shown in Wireshark. Now all you have to do is write a function that decrypts the data using the initialization vector and the key that was passed to the function that encrypts the data. I wouldn't suggest you to use Wireshark though. Fiddler is prefered when dealing with the HTTP protocol as it is much easier to use.
 

I am waiting for a Final Answer to these accusations as they seem very serious.
 
hatemachine said:
I am not sure why you are talking about MSIL. I was saying that those keys are removed to get rid of the Just In Time debugger which creates annoying windows and can screw with certain apps. Here's an example:

http://social.msdn.microsoft.com/Fo...ting-rid-of-justintime-debugger?forum=vsdebug

See the registry values? They're the same. Some have issues with them, and by removing those values you end up having less problems for your customers.

I am aware that RG can't properly decompile it, as it is a huge project and you'd have to change alot of the UI stuff to make it work. Can you actually compile it? I assume based on the bp's you set, that you can.



I don't see why it would take more than a few minutes? We know what encryption method is used ( DES ) and all you need is the initialization vector and the key which was used to encrypt the data, also you have the encrypted data shown in Wireshark. Now all you have to do is write a function that decrypts the data using the initialization vector and the key that was passed to the function that encrypts the data. I wouldn't suggest you to use Wireshark though. Fiddler is prefered when dealing with the HTTP protocol as it is much easier to use.
Click to expand...

1. See image below.
2. The program is more twisted than Chinese arithmetic. Somewhere between 60 and 80 percent of the program is devoted to encryption and all of it multi level redirection/polymorphism and nested functions.

gsbuild_zps2604a04a.png


Oh, Forgot:
msil - microsoft intermediate language.

Msil is what dotnet programs compile to.
 
Last edited:
JustUs said:
1. See image below.
2. The program is more twisted than Chinese arithmetic. Somewhere between 60 and 80 percent of the program is devoted to encryption and all of it multi level redirection/polymorphism and nested functions.

gsbuild_zps2604a04a.png


Oh, Forgot:
msil - microsoft intermediate language.

Msil is what dotnet programs compile to.
Click to expand...

Obviously the code is not going to be fully recovered, no one likes getting their work stolen. They used an obfuscator to scramble their code which is totally understandable from a developers point of view. You should try to reverse mine, you'd poke your eyes out; but that doesn't mean that I am hiding something. :) I know what MSIL is, but it has nothing to do with the debugger breaking certain apps. Anyway, to the main point; it is indeed strange that they would encrypt the URL and the Proxy. Care to post that code too? Certainly interested to hear the sellers response on this. Good find.
 
hatemachine said:
Obviously the code is not going to be fully recovered, no one likes getting their work stolen. They used an obfuscator to scramble their code which is totally understandable from a developers point of view. You should try to reverse mine, you'd poke your eyes out; but that doesn't mean that I am hiding something. :) I know what MSIL is, but it has nothing to do with the debugger breaking certain apps. Anyway, to the main point; it is indeed strange that they would encrypt the URL and the Proxy. Care to post that code too? Certainly interested to hear the sellers response on this. Good find.
Click to expand...

1. The man used the commercial edition of DotFusicator. Besides VS removing the symbols during a release build the commercial version of DotFusicator removes the few that VS leaves and then obfuscates what is left and adds dummy classes to further confuse the issue. It was not something that reverse engineered overnight! It was something that I was concerned enough that I spent roughly 30 days reverse engineering and about that long walking through it. Your point about the UI is on the money.

2. There is nothing that cannot be decompiled. It takes a lot of work, but if someone is sufficiently motivated it will be decompiled. We are not talking about cracking here.

3. I have my own homegrown scraper based on dotNet. If I attempt to run it when GScraper is running, it will throw an exception. I also have several other helper programs based on dotNet that fail if GScraper is running. The nature of a dotNet program is that .net uses hungry algorithms. At the same time, I have no problem running programs that use .net when run alongside a program that does not use .net.

4. While I could over the course of a couple of dozen posts show the code that encrypts the URL and proxys, I do think that it would be revealing more of Hawkes code than should be. So the answer is no. Let us see if Hawke fixes the problem.

5. I understand why people obfuscate code, esp. .net. That does not bother me. The other stuff I have seen does.

6. Notice the message on the bottom of the image under the error list, "Build Succeeded." The code, as you put it, is fully recovered.
 
Last edited:
JustUs said:
1. The man used the commercial edition of DotFusicator. Besides VS removing the symbols during a release build the commercial version of DotFusicator removes the few that VS leaves and then obfuscates what is left and adds dummy classes to further confuse the issue. It was not something that reverse engineered overnight! It was something that I was concerned enough that I spent roughly 30 days reverse engineering and about that long walking through it. Your point about the UI is on the money.

2. There is nothing that cannot be decompiled. It takes a lot of work, but if someone is sufficiently motivated it will be decompiled. We are not talking about cracking here.

3. I have my own homegrown scraper based on dotNet. If I attempt to run it when GScraper is running, it will throw an exception. I also have several other helper programs based on dotNet that fail if GScraper is running. The nature of a dotNet program is that .net uses hungry algorithms. At the same time, I have no problem running programs that use .net when run alongside a program that does not use .net.

4. While I could over the course of a couple of dozen posts show the code that encrypts the URL and proxys, I do think that it would be revealing more of Hawkes code than should be. So the answer is no. Let us see if Hawke fixes the problem.

5. I understand why people obfuscate code, esp. .net. That does not bother me. The other stuff I have seen does.

6. Notice the message on the bottom of the image under the error list, "Build Succeeded." The code, as you put it, is fully recovered.
Click to expand...

Awesome. Now we'll just have to wait for Hawkes! :)
 
Wow this have been turned into an amazing thread.let's see what happens
 
I've already send PM to Hawkes three days ago regarding the scraping urls issue. But till yet there is not any response from him.
I wonder if he would care to response here as well.
 
Well, what the actual fuck.

qPcaosb


I've scraped well over 100,000,000 URLs with Gscraper this month alone (I have their quarterly proxy service and have used them for a long time) and I've only had good things to say, but it has been going down from 250k links scraped/min to only around 100k and I've been seeing lagging issues as well (nothing changed on the dedi, other things like GSA, SB, CB, CS, etc work fine).

Let's see how this unwinds.
 
hatemachine said:
Well. The function encrypts string_0 and string_1. That doesn't say much without tracing the code back and checking what the result returned from smethod_3 is being used for. I don't see it being sent anywhere in that function, from where is smethod_3 being called? If someone has a valid license I could take a look and intercept the data. It would take like 10 minutes to verify the claims anyway.
Click to expand...
I have a license and the proxy service, shoot me a PM for my dedi information.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top