[New Feature] Beta Testing AI Detection for Unwanted AI Content

Hello everyone! Happy days abound! As of today we are officially Beta testing a new forum feature that will help detect unwanted AI content.

Why We Need an AI Detection System for BlackHat World

In the beginning (a little over a year ago), human written content outweighed AI generated content. BHW users were using it for translations, editing their thoughts or formatting long threads. This is all fine and perfectly acceptable ways of using AI. Sure, we prefer human written content but the world is changing and AI isn't going anywhere.

However, it's gotten out of hand. Over the past 12 months we've been tracking, recording and monitoring the use of AI to spam, increase post counts artificially and do other things to violate the Terms and Rules. And, you guys haven't been silent about it either. From users just wanting to stop with the AI translations (which will always be OK), to users noticing lots of AI generated spam, and declarations of the forum becoming unusable due to the AI replies. And lets not forget that AI slop is killing BlackHat World and it's killed SEO on BHW. Lets not forget the cries to get hard on AI generated drivel. There's been a lot more threads crop up, on average at least 2-3 a month since the AI content has become relatively common.

We know from the outside, it looks like we haven't been doing anything but from the process of monitoring and tracking the AI posts, we got the data we needed to make the next move. Build an AI detection system.

What is the AI Detection System and How Does it Work

It's exactly what it says it is. It's a feature we've built that helps detect the AI generated content that we don't want on the forum. Unfortunately, we can't give too many details on how we built it, so please don't ask. However, what we can tell you is the more you write yourself, the more likely the content is going to go through.

For those of you who insist on using AI to largely help with your threads and posts, they will be marked publicly as "suspected AI." From there other users can decide if they want to read what you have to say or if they're going to skip it completely.

To reiterate, the more you write that comes from your own head, the less likely your BHW content will get tagged as AI.

At this moment in time, the AI Detector is not issuing warnings. Any warnings that come due to AI generated content will come from someone on the Community Team who has seen and reviewed your thread or post. Keep in mind we aren't necessarily giving warnings because the content is AI generated, we're giving them because they broke the Terms and Rules. So if someone is using AI to spam, they'll get a warning for that. If they're trying to sell something, they'll get a warning for selling outside of the market place. If there's contact information, they'll get a warning for posting/quoting outside contact information. So on and so forth.

The AI Detection System Going Forward

I'd like to close with a few things:

1.) Please remember that this is in Beta. The tool isn't going to be perfect and will likely need some iteration and tweaking. So to sound like a broken record yet again, if you see something that you feel slipped through, use the report button. This will ALSO help us in tweaking the tool.

2.) Feel free to ask your questions, however, remember that we will not be answering questions about specifics on how this tool works.

3.) Our stance remains that using AI as a tool to assist in content creation is 100% ok. Using it to formulate your responses vs. your own voice, knowledge and expertise isn't.

We really hope that this helps in increasing the value of BHW while reducing the time the Community Team collectively spends on dealing with AI generated spam and garbage. We know nothing is 100% perfect so we still have a lot of work to do, but we feel that we're moving in the correct direction.

Please feel free to leave your thoughts, comments and feedback below!

Good move, this was overdue.

For suspected AI content do we have any filters on the form to prevent or remove those threads?

CodiBot said:

Good move, this was overdue.

For suspected AI content do we have any filters on the form to prevent or remove those threads?

Click to expand...

Not at the moment. However, it is on my wishlist for the future.

This a great. I do hope though that if there are false positives, there will be a way to report that so it gets corrected?

Brickbat1 said:

This a great. I do hope though that if there are false positives, there will be a way to report that so it gets corrected?

Click to expand...

Users can either report the post if it got published, or send in a ticket to support. However, as we've been testing this for over a month now on actual posts users have been making, we have the confidence that there aren't many false positives at this point.

I think this is a good direction because low-effort AI spam has definitely become a problem.

seems a good feature. a few minutes ago I tried to post in lounge and got under review and eventually my comment is being deleted. I was surprised to see that. May be beta version is the reason for this.

Wonderful news, mods are doing a great job fighting those bots

Hyfytab said:

What you need to do is incentivize users to participate and engage more

Click to expand...

You mean incentives such as reaching 15 posts and being able to send DMs and use certain areas of the forum?

Hyfytab said:

requiring 15 posts just to access certain areas of the forum feels more like a barrier than an incentive

encourages users to post just for the sake of numbers rather than meaningful contributions and it's creating that pressure to spam for access

Click to expand...

Well with an AI detector in place, they will be encouraged to post good content to access those features. Nobody is going to do business with someone who lazily slops their way to 15 (or 100) and has no reputation to speak of. If they're serious businesspeople then they will put the effort in.

Sounds great, and was absolutely needed. It’s great to see bhw trying to keep the user experience and content top tier

I literally saw a thread the other day where 80% of the comments were new accounts saying exactly the same thing but with different words, it was so blatantly AI with literally 0 value that I just felt like closing the forum or the rest of the day after seeing so many similar posts in other threads. I'm glad this is finally happening. I think AI threads is the least of our concern personally, the biggest issue is the people spamming AI posts on threads just stating the obvious

Hyfytab said:

I get the perspective about sellers; yes they need to establish credibility.

however, why should I, as a buyer, have to meet a post threshold before I can engage?

ultimately, it's the seller's choice whether they want to work with me or not.

it would be more beneficial to allow potential buyers to participate freely from the start and post their project in hiring section

Click to expand...

You're a new poster, and as such you wouldn't have seen the state of those sections before we put them behind a 15 post barrier. They were flooded with bogus proposals and offers, tons of junk spam and general other detritus. That's why the barrier was put in place and it has significantly improved those sections and reduced the number of disputes we've seen from threads in them.

Hyfytab said:

fair enough, but isn't this reducing overall traffic? as a result, you see fewer disputes

leading to an increased usage of AI as they try to reach 15 posts

I am trying to say that I see the usage of AI as a counter effect to the fact that the new user experience is very limited and restricted.

Click to expand...

AI-generated posting was a major discussion point on the forum before the 15 post rules were brought in. Those rules have had a negligible impact on how much it is used. Its increase is due to the rapidly growing numbers of AI platforms that people have access to, and their increased ease of use.

You also seem to be ignoring the certainty that people will be attracted to the forum, return to it, or increase their engagement when they know they are far more likely to receive, and interact with, human responses.

Hyfytab said:

because that doesn't change the fact that the new user experience feels very limited and restrictive and many will leave just as quickly as they came

from your perspective, what is the incentive for a buyer or new user to stay here?

why shouldn't they just go to Fiverr, get what they need to get done, and receive a couple of offers within minutes, rather than grinding for a week or two just to be able to post here?

as far as I understand, this forum generates revenue from seller subscriptions and advertising spots. so, isn’t it in your interest to keep buyers engaged here?

Click to expand...

The people who really see value in this forum will stay, and new users usually come here because they hear BHW has value and real knowledge inside it. I was actually reading information from this forum through Google long before I even registered here, probably before 2010, and that is exactly how I saw the value of BHW. The only shame is that I registered only last year.

Hyfytab said:

because that doesn't change the fact that the new user experience feels very limited and restrictive and many will leave just as quickly as they came

Click to expand...

Our statistics show that is not the case. Also, it is more or less the norm that a forum has restrictions on features for new members to encourage them to be a part of the community and incentivise their participation.

Hyfytab said:

from your perspective, what is the incentive for a buyer or new user to stay here?

Click to expand...

I've already answered this a couple of times now.

Hyfytab said:

why shouldn't they just go to Fiverr, get what they need to get done, and receive a couple of offers within minutes, rather than grinding for a week or two just to be able to post here?

Click to expand...

I have no experience of Fiverr so I can't compare.

Hyfytab said:

as far as I understand, this forum generates revenue from seller subscriptions and advertising spots. so, isn’t it in your interest to keep buyers engaged here?

Click to expand...

Again, this has already been answered. Members will be more engaged in a forum that offers overwhelmingly human content.

While I do use AI to help with productivity, because it's a tool and I subscribe to the work smarter not harder tribe, I do think it's made people extremely lazy. I've even noticed it myself in some things that I use it for and I've had to make a conscious effort to do things more by hand in order to not lose skills. We didn't have this problem with laziness a few years ago, we were all forced to either write our own copy, hire someone to do it for us, or copy it and hope nobody noticed. So if new users honestly think that coming up with content on their own is "too hard" vs. entering a thread title or question and simply copying/pasting the answer (not even attempting to make it look human or fact checking it) then I'm sorry. It's not the direction that we want to go.

This was a must feature.

PetraAnn said:

(not even attempting to make it look human or fact checking it)

Click to expand...

You are right on that part. What surprises me most is not even the use of AI itself, but how many people just copy and paste whatever it gives them without checking if it even makes sense.

Thank you to the BHW team for this much needed feature. This is the best news I could have had today!