Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forum Operator #19

Open
cbeams opened this issue Sep 4, 2017 · 89 comments
Open

Forum Operator #19

cbeams opened this issue Sep 4, 2017 · 89 comments
Assignees
Labels
team:ops https://bisq.wiki/Ops_Team

Comments

@cbeams
Copy link
Member

cbeams commented Sep 4, 2017

This role is about operating and administering the Discourse forum at https://bisq.community.

Spec: https://github.com/bisq-network/roles/blob/master/forum-operator.adoc

@cbeams
Copy link
Member Author

cbeams commented Jan 2, 2018

@alexej996, I've just assigned you to this role, reflecting the fact that you're actually doing this work.

Could you please do a monthly update comment here for December, per #57 (comment).

Specifically, could you please address the two recent outages of the forum, and what you plan to do to prevent these in the future? I think there are a couple major issues here:

  1. Your ability to see and respond to outages in the first place. In both cases, it took several hours or more to get hold of you.

  2. Getting a backup/secondary operator in place. I believe @Emzy now has access to the Discourse server as well as admin rights, but it's not clear to me whether he's actually signing up to be on call for this, i.e. watching the #forum channel, watching bisq-network/forum issues, etc.

Thanks!

@alexej996
Copy link
Member

2017.12 update

Discourse forum was upgraded at least once this month during the second forum outage on 27th of December to the current latest version 1.9.0.beta17.
Backups were done automatically every two days as per previous Discourse configuration, but I downloaded them to a local computer at least once a week in case something goes wrong.

There were two outages of the forum for still unknown reasons. First one was on 7th of December and second was on 27th of December. Both happened during work days while I wasn't available until the evening. In future I will try making a bash script to automate restarts of Discourse in case forum is down.
I will also ask @Emzy to be backup/secondary operator for a portion of the BSQ compensation, if he is not available for the role I will try finding someone else.
I already installed today Slack on my phone so I can be available hopefully at all times on the #forum channel.

This month we had increase in users and user visits as well as posts and topics.
There was 1151 user visits this month, which is 33% increase from last month.
We had 283 new users, that makes 180% increase from last month.
Number of new topics created this month was 181. That is 121% increase.
Number of new posts was 913 with 78% increase from last month.
In total there was 137,845 page views this month with 110% increase from last month.

@cbeams cbeams changed the title Discourse forum operator Forum Operator Jan 2, 2018
@cbeams cbeams added role and removed role:op labels Jan 3, 2018
@ManfredKarrer
Copy link
Member

@alexej996 Great thanks for the update.
@Emzy Are you available as backup for the forum?

@Emzy
Copy link

Emzy commented Jan 3, 2018

Yes I'm available as backup for the forum.

@cbeams
Copy link
Member Author

cbeams commented Jan 8, 2018

Thanks, @Emzy. @alexej996, could you work out with @Emzy what being a "backup" means and add it to the role spec? Doesn't have to be elaborate. Should just make it clear what the responsibility of the backup is, when and to what they are expected to respond, etc.

@Emzy, I've added you as an assignee here to indicate that you're the 'backup' or 'secondary' role owner here (we can figure out that terminology as we go here). Thanks, both.

@alexej996
Copy link
Member

2018.1 update

Discourse forum was updated once this month to the currently latest version 2.0.0.beta1 on 5th of January.
Backups were downloaded at least twice per week as described in the role specification.

The forum experienced no issues or down time this month and seems to be running as intended.

We had an increase in user visits this month and a decrease in new users, topics and posts, as well as a decrease in total page views.
There were 1217 user visits this month, which is 8% increase from December.
We had 267 new users join our forum, which is around 5% less than last month.
135 topics where created this month. That is a 27% decrease from last month.
839 posts where created in January, which is 10% less than December.
In total there were 114261 page views this month, which makes 19% decrease form the last.

@alexej996
Copy link
Member

2018.2 update

Discourse forum was updated twice this month, on 3rd of February to version 2.0.0.beta2 and to version 2.0.0.beta3 on 16th.
Backups were downloaded at least twice per week as described in the role specification.

On 9th of February SSL certificates for the old forum domain (forum.bisq.io) were manually renewed.

On 19th of February there was a temporary issue with the SSL certificates for the new domain, as they weren't renewed before expiry. These SSL certificates were automatically installed by Discourse software, there were no warnings of their expiry nor did the software renew them automatically.
The issue was resolved in few minutes when I installed them manually after being notified of the issue.
In future there should be a warning for their expiry, just like there is one for certificates for the old domain.

Operating system was updated on 19th of February after SSL certificates were renewed.

There was an increase in response time to topics in the last month, but decrease in overall activity on the forum.
There were 909 user visits in the last month, which is 25% decrease.
108 new users in February, which is 59% decrease.
79 topics, which is 41% decrease.
436 posts made, which is 48% decrease.
102,202 total page views this month, which is 13% decrease from January.

@alexej996
Copy link
Member

2018.3 update

Discourse forum was updated once this month, on 8th of March to version 2.0.0.beta4.
Backups were downloaded at least twice per week as described in the role specification.

There were some issues with forum sending emails this month. Gmail seems to be suspicious of SMTP logins from the forum.
There were also some phishing attempts that started this month. Emails are being sent to the forum's email address abusing SMTP login issue to appear trustworthy. Not sure if it these issues are connected in any way.
Perhaps Gmail is suspicious of SMTP logins because of possible attempts by attacker to login over SMTP to either guess the password or create enough suspicion to cause Gmail temporarily locking an account. Either way, I didn't fell for the phishing emails, but the forum maybe still experiences some email login issues due to high suspicion. I will look into it in the following month.

On 26th of Match forum was migrated to the new bigger server.
Switch was made from an AWS hosting to Namecheap, as they support Bitcoin payments, which I prefer of course.
Forum should also be capable now of handling a bigger community as we will hopefully have soon.

There was a decrease in forum activity this month as well. This month we had:
612 user visits, 32% decrease.
107 new users, pretty much the same as last month.
56 topics, 33% decrease.
319 posts, 26% decrease.
80,817 total page views, 24% decrease.

@alexej996
Copy link
Member

2018.4 update

Discourse forum has been updated twice this month. First time on 13th from version v2.0.0.beta4 to v2.0.0.beta6 and second time on 27th to currently latest version v2.0.0.beta7.
Backups were downloaded at least twice per week as described in the role specification.

We had more issues with emails this month, although not quite as severe. On 5th of April forum was reconfigured to use Mailjet for sending emails, instead of Gmail.
We currently have an issue of having an hourly limit on how many emails the forum can send (bisq-network/forum#5), but it seems like we will resolve it soon.

This month activity on the forum decreased. We had:
433 user visits, which makes a 37% decrease.
91 new users, making an 11% decrease.
44 topics were created, which is a 36% decrease.
248 posts were made, which is a 35% decrease.
71225 total page views, which is a 20% decrease.

@alexej996
Copy link
Member

alexej996 commented Jun 22, 2023

Cycle 49 report

Screenshot_2023-06-22_12-24-44

Forum activity decreased this cycle.

/cc bisq-network/compensation#1327

@alexej996
Copy link
Member

alexej996 commented Jul 22, 2023

Cycle 50 report

Screenshot_2023-07-22_11-00-30

Activity increased this cycle back to usual levels.

/cc bisq-network/compensation#1348

@alexej996
Copy link
Member

alexej996 commented Aug 21, 2023

Cycle 51 report

Screenshot_2023-08-21_13-44-43

Activity is similar to last cycle.

/cc bisq-network/compensation#1359

@alexej996
Copy link
Member

alexej996 commented Oct 4, 2023

Cycle 52 report

Screenshot_2023-09-22_00-59-00

Activity is similar to last cycle.

/cc bisq-network/compensation#1390

@alexej996
Copy link
Member

alexej996 commented Nov 3, 2023

Cycle 53 report

Screenshot_2023-11-03_12-39-03

Forum activity is the same as last cycle.

/cc bisq-network/compensation#1413

@alexej996
Copy link
Member

alexej996 commented Dec 9, 2023

Cycle 54 report

Screenshot_2023-12-09_23-51-01

Forum activity is less than half of what it was last cycle.

/cc bisq-network/compensation#1436

@alexej996
Copy link
Member

alexej996 commented Jan 11, 2024

Cycle 55 report

Screenshot_2024-01-11_13-38-10

Forum activity further decreased from last cycle by about 30%.

/cc bisq-network/compensation#1460

@alexej996
Copy link
Member

alexej996 commented Feb 10, 2024

Cycle 56 report

Screenshot_2024-02-10_01-05-40

Forum activity tripled from last cycle.

/cc bisq-network/compensation#1480

@alexej996
Copy link
Member

alexej996 commented Mar 12, 2024

Cycle 57 report

Screenshot_2024-03-12_14-32-14
Forum activity stayed the same from last cycle.

/cc bisq-network/compensation#1500

@cbeams
Copy link
Member Author

cbeams commented Apr 13, 2024

Attention @bisq-network/forum-operators: https://bisq.community is down and has been for at least the last two hours.

(screenshot from #Ops Matrix channel)
image

@suddenwhipvapor
Copy link

No developments since the above?

@alexej996
Copy link
Member

Cycle 58 report

There was a huge problem with the VPS that was hosting the forum and unfortunately all data since the last backup was lost. Last backup that was stored offline on my personal computer was long time ago, from last year.
This is a huge problem and unfortunately no way to get the data back.

The forum is now restored to that last backup and is back online.

@alexej996
Copy link
Member

alexej996 commented May 27, 2024

For Cycle 59

Screenshot_2024-05-27_12-29-49

Forum activity seems to have increased this cycle.

/cc bisq-network/compensation#1548

@alexej996
Copy link
Member

alexej996 commented Jul 3, 2024

For Cycle 60

Screenshot_2024-07-03_03-50-02

Forum activity slightly decreased from last cycle.

/cc bisq-network/compensation#1573

@cbeams
Copy link
Member Author

cbeams commented Jul 13, 2024

Hi @alexej996, could you comment on the current state of backups for the Forum? I may have missed it, but I don't believe this was ever followed up on after the outage in April. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team:ops https://bisq.wiki/Ops_Team
Projects
None yet
Development

No branches or pull requests

6 participants