Dev Diary – Updated 11/07/2018

[timchubbKeymaster 3166xp 12]

11-07-2018

Well we have had a busy week and a bit since the last dev diary update….

Today we have launched the first (of many) minor point releases, yup the site is officially now 2.0.1!  On the original roadmap this was to contain updates to notifications (emails mainly) and the old forums, unfortunatly due to the performance issues we encountered thats not what the meat of this release is about.

In 2.0.1 we have the following:

• Home page layout tweaks – @lloyd has made some really slick slideable content which works a treat on mobile devices, such as the latest project update area on the home page, oh yeah projects, place and games now all have a place on the home page as well 😀
• Massive optimisation and caching on the core pages of the site, the root cause of the performance issues was some unoptimised database and page logic, this has been rewritten and optimised to only run when necessary.  In order to side step the performance issue we threw hardware at it, but this is a very short term fix for something that can be and has been fixed in code, in terms of effciency increases we have gone from over 300 separate database queries to load the home page to approx 28 for most requests, and i have started to scale down the frontend cluster to double check we can handle the load on our preferred server size.  We have also ensured that the cache is as long lived as possible thus reducing the frequency of full page loads which hit the database harder.  The upshot of this is point 3…
• Realtime updates to the user content driven parts of the home page!, yup comment counts, new places, project activity, game reviews, game ratings and forum activity all now trigger the home page to update no longer is there a lag between activity and it showing.

Roadmap

Fixed

• Lack of caching

In Progress

• Forum notifications, first cut will just be for new topic creation and subscribed threads.
• Tweaks to project image uploads, hope to resolve the issues people have been having with failed uploads.
• SEO Improvements
• Rich card support for linked to BoW content

Upcoming

• Forum Archives
• Better notification management

We will be monitoring server performance but fingers crossed from now on its back to feature development… 🙂

01-07-2018

We have had a bit of a blip in site performance starting approx 3pm GMT 30/06/2018, bizarrely it was effecting only one of the servers in the front end pool, but sods lawing what it is, it was the favoured node of the load balancer, so it was getting the majority of traffic routed to it.  Further analysis showed that there was a problem with worker process recycling, when you request a page behind the scenes various calls are made to databases and other systems, and in a perfect world, the site always tidies up after it self closing connections etc., however thats not the world we live in, so the worker processes should get fixed by getting effectivly “turned off and on again” this wasnt happening, so i have altered the thresholds for this virtual off and on to happen and it seems to have improved the situation.

Onwards we are monitoring the processes and shall be revisiting these thresholds when i have more traffic pattern data (i.e. what happens when you all flock to the site on a sunday morning for your breakfast xlbs 😉 )

 

28-06-2018
Hi Guys

First of all sorry for the slow performance of the site for the last week or so, unfortunately its all part of the tuning and fettling process post go live of 2.0. For those interested, the root cause was a problem with an internal DNS server incorrectly routing DB requests to a public interface rather than one on the virtual LAN that connects the cluster together. But it made us focus on some performance tweaks and changes to our cache mechanism and how we implement certain bits of functionality, which now only run when there is a change, rather than for every page request even if the functionality is not used on the page.

Now for the fun part the list of what’s been fixed, what’s in progress, and what’s upcoming:
Fixed

• Session state is shared between front end servers, which should mean that people finding them-selves logged out should stop happening, and the remember me checkbox on the login page should work as advertised.
• Lots of DB optimisation, successfully shrunk the daily slow query log from 150mb a day, to 500k, which is a not too shabby 99.6% decrease in slow queries
• Site performance, as mentioned a DNS SNAFU caused the massive slow down on page requests, that’s now resolved, pages should load between 1~10 secs now on the front end of the site, even quicker if you are signed out and hitting the page cache.
• Mail feed generator has been upgraded to work with 2.0 data structures, so expect the newsletters to resume in earnest once @warzan irons the last of the style kinks out.
• Image resizer library has been fettled for speed and performance.

In Progress

• Forum notifications, first cut will just be for new topic creation and subscribed threads.
• Tweaks to project image uploads, hope to resolve the issues people have been having with failed uploads.
• SEO Improvements
• Rich card support for linked to BoW content

Upcoming

• Forum Archives
• Better notification management

Keep on finding the bugs guys, the list is getting shorter but im sure there are some more lurking 😀

[davehawes 7768xp 0]

Thanks for all the efforts and best of luck with ongoing bug squashing.

[a27cromwell 3186xp 1]

No idea what you said , except that your fixing things , and for that I thank you and the rest of the crew at head office

[warzanKeymaster 30960xp 1]

@tgu3 also fixed a bug we noticed in achievements related to projects. It was giving achievements correctly for folk giving them but not for the folk receiving them.

 

It’s fixed now but obviously there are members who have a lot of nominations who haven’t unlocked achievements yet as the count is only starting from now for everyone.

I have a small dilemma around this in that we could explore retrospectively adding this but I suspect a week of dev time could easily be sunk into that and I’m reluctant to slow down the overall roadmap of fixes and improvements we are trying to burn through.

Does anyone have any strong feelings on this either way?

[timchubbKeymaster 3166xp 3]

@warzan shall see how possible a dB query will be…

[jamjarst 4011xp 0]

Great work @timchubb and everyone else involved. Just in case you don’t hear enough positive feedback but I wanted to add my praise for the new site. I much prefer it to the old one which I was not a great fan of and the Project section is absolutely brilliant?

[soapdodger 17313xp 2]

@warzan I don’t have strong feeling about it people who use it will get the achievements eventually. Though I only noticed this when I logged in and got some achievements.

I put both of my projects on hold so I can start to think about the hobby challenge,it bumped them to back to the top of the project list, same when I corrected spelling last week.

This is quite odd, I would have thought it would have only classed an update as a new post. Now I feel a bit bad as people may click for no new content.

Only realised this due to the achievements.

[warzanKeymaster 30960xp 1]

Good spot @soapdodger we may have to make the code a little smarter in that area

 

Thanks for the valuable feedback mate

[timchubbKeymaster 3166xp 0]

bump

[warzanKeymaster 30960xp 0]

Really really pleased with today’s  (pretty massive) update @timchubb the homepage seems to be operating beautifully so far 🙂

Nice work @tgu3 @lloyd 🙂

[horati0nosebl0wer 19611xp 1]

Oooo… shiny tech babble. I might not be cleared for this Ultraviolet level stuff. Being a simple Red sector fellow (possibly cleared Orange the longer I stay on the boards) I am confused by the Computer. I know it is my friend and I duly serve it.
{Paranoia refs for those thinking I’m more than a little off here}

Good stuff I’ll say. The performance has picked up for the mobile use when I check the site while at work.

[wahidovicParticipant 10xp 0]

no thing is better than when we thanks the other

thank you

[horati0nosebl0wer 19611xp 1]

A little odd but after seeing the server was busy I resent a topic for the forum and the earlier entry popped in. Just wondering if you guys were still tinkering and threw some things together (or uploaded a video) and wanted to let you know.

[timchubbKeymaster 3166xp 1]

Always tinkering ?

[ameliasParticipant 10xp 0]

thanks for the improvements and good luck !

[Author]

Posts