spam?
Is this good for me? Don't feel like.
> select count(*) from gserver where url LIKE '%troll.cf%';
+----------+
| count(*) |
+----------+
| 13837755 |
+----------+
1 row in set (1 min 20.495 sec)
| 172846 | 30m1uebec.activitypub-troll.cf | 30m1uebec.activitypub-troll.cf | | | | 0 | 0 | | | unkn | | 0 | | 2022-12-03 19:51:17 | 0001-01-01 00:00:00 | 2023-01-03 20:01:25 | 0001-01-01 00:00:00 | 0 | 0 | 0 | 2023-02-03 20:01:25 | NULL | NULL | NULL | NULL | NULL | NULL |
| 172847 | 1ml1up799.activitypub-troll.cf | 1ml1up799.activitypub-troll.cf | | | | 0 | 0 | | | unkn | | 0 | | 2022-12-03 19:51:18 | 0001-01-01 00:00:00 | 2023-01-03 20:01:26 | 0001-01-01 00:00:00 | 0 | 0 | 0 | 2023-02-03 20:01:26 | NULL | NULL | NULL | NULL | NULL | NULL |
| 172848 | 2ckkegfqs.activitypub-troll.cf | 2ckkegfqs.activitypub-troll.cf | | | | 0 | 0 | | | unkn | | 0 | | 2022-12-03 19:51:20 | 0001-01-01 00:00:00 | 2023-01-03 20:01:28 | 0001-01-01 00:00:00 | 0 | 0 | 0 | 2023-02-03 20:01:28 | NULL | NULL | NULL | NULL | NULL | NULL |
| 172849 | q2g4bs0i.activitypub-troll.cf | q2g4bs0i.activitypub-troll.cf | | | | 0 | 0 | | | unkn | | 0 | | 2022-12-03 19:51:21 | 0001-01-01 00:00:00 | 2023-01-03 20:01:28 | 0001-01-01 00:00:00 | 0 | 0 | 0 | 2023-02-03 20:01:28 | NULL | NULL | NULL | NULL | NULL | NULL |
Michael Vogel
in reply to grin • • •*.activitypub-troll.cf
.Jonas ✅
in reply to grin • • •Yes, I had the same problem. Just block the domain.
If your instance crashes because of too many database entries block the subdomains in chunks: github.com/friendica/friendica…
@Friendica Support
Lorenz
in reply to grin • • •Jonas ✅
in reply to Lorenz • • •Roland Häder
in reply to Lorenz • • •./bin/console serverblock add <pattern> <reason>
Roland Häder
in reply to Roland Häder • • •Lorenz
in reply to grin • • •bin/console serverblock add *.activitypub-troll.cf spam
grin
in reply to Lorenz • • •Lorenz
in reply to grin • • •grin
in reply to Lorenz • • •Lorenz
in reply to grin • • •Lorenz
in reply to grin • • •Roland Häder
in reply to Lorenz • • •contact
andgserver
isON DELETE RESTRICT
which is a security measurement, that you do not delete accidentally something and then ending up with a nearly empty database. So you have to delete all those records there, too. Please do not change the constraint toCASCADE
.Lorenz
in reply to Roland Häder • • •Query OK, 13779424 rows affected (1 hour 16 min 19.826 sec)
Roland Häder
in reply to Lorenz • • •DELETE
statement. The statement ran through because no records incontact
linked to any of those (see columngsid
).Lorenz
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •Roland Häder
in reply to Roland Häder • • •Lorenz
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •OPTIMIZE TABLE `name`
is a good start. MariaDB does change that for InnoDB tables (like Friendica uses) to recreating the table and copying all rows over then analyzes the table for gathering statistics:stackoverflow.com/questions/30…
Lorenz
in reply to Roland Häder • • •Lorenz
in reply to Roland Häder • • •OPTIMIZE TABLE gserver;
and it deleted more than 3GB!Roland Häder
in reply to Lorenz • • •contact
records from blocked domains and then purges them from thegserver
table.grin
in reply to Roland Häder • • •So no, optimize table doesn't do anything for innodb. Copying/renaming is painful for huge tables.
Correction: after removing (better) optimize started, and recreated in a flash. Thanks!
Roland Häder
in reply to grin • • •Anders Rytter Hansen
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •Lorenz
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •grin
in reply to Roland Häder • • •Lorenz
in reply to grin • • •grin
in reply to Lorenz • • •Roland Häder
in reply to grin • • •OPTIMIZE TABLE `gserver`
and it will be recreated+analyzed for you.Roland Häder
in reply to grin • • •code
block?Anders Rytter Hansen
in reply to grin • • •Lorenz
in reply to grin • • •Roland Häder
in reply to Lorenz • • •grin
in reply to grin • • •I have banned and purged sbcloud.cc from everywhere, based on this
Since then worker doesn't pull in spambots again.
Now, it would be neat to know:
1. What exatly happened (I don't know the protocol that deeply)
2. Who did what
3. How to prevent that from happening in the future (both network-wise and locally)
#spambot #spam
grin
in reply to grin • • •I am not sure I'll ever grok how this is supposed to work, who gets notified when and who see what where how.
Roland Häder
in reply to grin • • •sbcloud.cc
looks very legitimate to me, no sign of spam/scam. Please take a look at this:So Even the domain
activitypub-troll.cf
doesn't exist, butsbcloud.cc
does. I see no relation between these two domains as they share nothing in common.grin
in reply to Roland Häder • • •Roland Häder
in reply to grin • • •sbcloud.cc
.grin
in reply to Roland Häder • • •Why? You think that having dns is proof that no bad traffic comes from there? Especially since you seem to realise that the spammed addresses were fakes, yet you seem to expect "blocking" a non-existent server. You based your opinion on about zero amount of facts, but you seem to be quite assured that you are, somehow, right.
But anyway, stopped spam for me, you're free to do whatever you deem proper, including looking at the dns when the AP networks get abused. :shrug:
I wish there were useful logs: those would be better for abuse management than... dns.
Roland Häder
in reply to grin • • •NXDOMAIN
) domain originating from their IP address?Lorenz
in reply to grin • • •Even after I have blocked these servers more than two weeks ago, the gserver table had more than 8GB! Now I run the same delete command again, and the table now has 10GB. What happened? Somebody knows what to do? Weird stuff.
UPDATE: I run
OPTIMIZE TABLE gserver;
- and now, wow! the table is nearly empty, just 31 MB, and now it seems I did not have to upgrade my VPS!Roland Häder
in reply to Lorenz • • •@Lorenz @grin We need to add this to the purge worker job to have it automatically done for you. Or maybe optimize all tables? Just a
SHOW TABLES FROM `friendica`;
and exlude all views? A query onmysql
schema is to much "vendor-specific" and the script already "knows" which are views and which are tables.PS: Your both avatars are not showing up here, even after a "Refetch contact data".
Lorenz
in reply to Roland Häder • • •I tried to optimize all tables, but that lasted too long, so I stopped it.
I am surprised to hear that the avatar is not showing. What can be the reason? What can I do?
Roland Häder
in reply to Lorenz • • •$ screen -dmS mysql mysql -p -u <user> <database>
And insert your data. Please don't include your password in the parameter list as this is visible withps -ax
. Then you can let the optimization run. You can access it withscreen -r mysql
and leave it withALT+AD
(exact order!) without quitting it.Lorenz
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •2023-02-13T20:26:39Z worker [ERROR]: Uncaught exception in worker execution {"class":"Friendica\\Core\\Storage\\Exception\\StorageException","message":"Database storage failed to update ","code":500,"file":"/var/www/.../src/Core/Storage/Type/Database.php:94","trace":"#0 /var/www/.../src/Model/Photo.php(449): Friendica\\Core\\Storage\\Type\\Database->put()\n#1 /var/www/.../src/Model/Photo.php(636): Friendica\\Model\\Photo::store()\n#2 /var/www/.../src/Model/Contact.php(2307): Friendica\\Model\\Photo::importProfilePhoto()\n#3 /var/www/.../src/Model/Contact.php(2792): Friendica\\Model\\Contact::updateAvatar()\n#4 /var/www/.../src/Model/Contact.php(2585): Friendica\\Model\\Contact::updateFromProbeArray()\n#5 /var/www/.../src/Worker/UpdateContact.php(47): Friendica\\Model\\Contact::updateFromProbe()\n#6 [internal function]: Friendica\\Worker\\UpdateContact::execute()\n#7 /var/www/.../src/Core/Worker.php(572): call_user_func_array()\n#8 /var/www/.../src/Core/Worker.php(386): Friendica\\Core\\Worker::execFunction()\n#9 /var/www/.../src/Core/Worker.php(121): Friendica\\Core\\Worker::execute()\n#10 /var/www/.../bin/worker.php(83): Friendica\\Core\\Worker::processQueue()\n#11 {main}","previous":"Exception: Got a packet bigger than 'max_allowed_packet' bytes in /var/www/.../src/Core/Storage/Type/Database.php:94\nStack trace:\n#0 /var/www/.../src/Model/Photo.php(449): Friendica\\Core\\Storage\\Type\\Database->put()\n#1 /var/www/.../src/Model/Photo.php(636): Friendica\\Model\\Photo::store()\n#2 /var/www/.../src/Model/Contact.php(2307): Friendica\\Model\\Photo::importProfilePhoto()\n#3 /var/www/.../src/Model/Contact.php(2792): Friendica\\Model\\Contact::updateAvatar()\n#4 /var/www/.../src/Model/Contact.php(2585): Friendica\\Model\\Contact::updateFromProbeArray()\n#5 /var/www/.../src/Worker/UpdateContact.php(47): Friendica\\Model\\Contact::updateFromProbe()\n#6 [internal function]: Friendica\\Worker\\UpdateContact::execute()\n#7 /var/www/.../src/Core/Worker.php(572): call_user_func_array()\n#8 /var/www/.../src/Core/Worker.php(386): Friendica\\Core\\Worker::execFunction()\n#9 /var/www/.../src/Core/Worker.php(121): Friendica\\Core\\Worker::execute()\n#10 /var/www/.../bin/worker.php(83): Friendica\\Core\\Worker::processQueue()\n#11 {main}","worker_id":"cec5e9a","worker_cmd":"UpdateContact"} - {"file":"Worker.php","line":577,"function":"execFunction","request-id":"63ea9ba037f4d","uid":"a61a92","process_id":21743}
Lorenz
in reply to Roland Häder • • •Roland Häder
in reply to Lorenz • • •Lorenz
in reply to Roland Häder • • •Exception: Got a packet bigger than 'max_allowed_packet' bytes
Seems to be on your end then?
Lorenz
in reply to grin • • •Two months later same issue:
more than 9GB freed up!
Raroun
in reply to grin • • •Running 2023-03-rc on the last commit.
86k server from *.gab.best.
select count(*) from gserver where url LIKE '%troll.cf%' OR `url` LIKE '%gab.best%';
+----------+
| 86378 |
+----------+
DELETE FROM `gserver` WHERE `url` LIKE '%activitypub-troll.cf%' OR `url` LIKE '%gab.best%';
Query OK, 86378 rows affected (1.143 sec)
Changed Block pattern from gab.best to *.gab.best.
Obiviously i missed the wildcard.
Lorenz
in reply to Raroun • • •The thing is I have added the wildcard and blocked the other troll-domain, and nevertheless, I still get all their spam.
so within one week the result:
Roland Häder
in reply to Lorenz • • •gserver
table, I wonder why is that happening? Someone must have introduced a bug or has removed it. My version here on my instance is working perfectly fine.Roland Häder
in reply to Lorenz • • •SELECT * FROM `gserver` WHERE `url` LIKE '%activitypub-troll.cf%' OR `url` LIKE '%gab.best%';
gives me zero rows here.OldKid ⁂
in reply to Lorenz • • •@Lorenz Please block only *.activitypub-troll.cf the other blocks should not do anything.
@Roland Häder if I remember correctly your fix was added to the 2023.03-rc branch. The instance of @Lorenz runs on 2023.01, so still without the fix.
Lorenz
in reply to OldKid ⁂ • • •Alright, I will upgrade to the RC the coming days, then. Thanks!
@Roland Häder @Friendica Support
Roland Häder
in reply to OldKid ⁂ • • •*troll.cf
is to much.Lorenz
in reply to grin • • •Raroun
in reply to grin • • •The pull request is marked in the 2023-03 Milestone, so I guess its in the actual RC and later in 2023-03-stable.
Link to pull request #12700