The Monthly Economic Report for October 2020 is now available!
You can download all of the raw data used in this report here. As always, each image can be enlarged by clicking on it.
Greetings, performance-hungry Capsuleers!
New Eden is at war again, and with war come new World Record-breaking numbers. An amazing 6557 pilots were in a single system at the same time, although a staggering 8825 players were involved overall.
Signature to the war are the massive fleet battles which represent opportunities for performance measurement. Valuable knowledge is gained by collecting client measurements during these fights.
Battles in EVE Online are incredibly dynamic and unpredictable. Your client does not know what ships will be jumping into the system until they enter, so it must load assets at that point. There are a number of assets that need to be loaded for every player including ship model, textures, sounds, weapon models, animations, and visual effects. With hundreds of different ships, SKINs, weapons, and a plethora of ammo types, it is no wonder that the EVE client wants plenty of memory.
As part of the continuing work to strengthen the foundations of EVE as it goes into its third decade, a further step is being taken to improve client memory performance. While these memory improvements will benefit everyone, they help more dramatically in situations where there will be many different assets loaded by the client such as the large fleet fights mentioned above.
Since large fleet battles are not always available when needed, specific stress tests have been built. The test below has been affectionately named the “Cube of Death”:
The “Cube of Death” test features 1000 ships that are evenly spaced and stationary. This test facilitates taking reproducible performance measurements between changes – allowing the comparison of before and after results. This particular test shows off the new memory improvements very well, as the most dramatic improvements come from busy scenes with lots of assets.
Let's talk numbers!
There are two types of memory that EVE uses: GPU and System. When people discuss “RAM” they are generally talking about system memory, but it is important to understand the difference.
GPU memory is used to store scene textures, meshes, and other graphics-related data. In a normal desktop PC with a dedicated graphics card, GPU memory is located on the graphics card itself. When you choose higher graphical settings in a game, it will use more GPU memory as a result.
System memory is used to contain everything else the game needs to run: the Python interpreter, the user interface, sounds, networking, user input from the keyboard and mouse, and the localization engine.
While both GPU and System memory will see an improvement to memory use, the big winner with this change is system memory.
In the current version of the EVE client, the “Cube of Death” scene uses about 3600MB of system memory. After the change, it has dropped to approximately 3000MB, or around a 17% decrease in memory use by the client.
The amount of memory improvement is highly dependent on the scene, but the decrease in System memory use is present no matter if there is one ship or thousands. It will just be more noticeable in scenes that are complex and diverse in nature.
The continuing investment in new technology and processes such as the introduction of the 64-bit client, improved crash reports, and now the RAM reduction will set EVE up strongly for its third decade. While much has already been done towards realizing this goal, there are still exciting opportunities and projects still to come.
Large scale fleet fights in EVE and the significant player wars that take place in New Eden are important to EVE, and this latest improvement is another step towards improving the experience and performance for all players involved in making EVE history, while also writing its future.
New Eden is a complex universe with huge player-driven fights, a vast array of gameplay options, a massive player-controlled market, wormhole civilizations, and much more. Striving to find in-game advantages, players are constantly trying mechanics and interactions which were unplanned and emergent.
The broad range of player computer configurations have a similar complexity. Operating system versions (including patches), individual system configuration changes, driver versions, and the reliability of the underlying hardware all play a critical role in how the EVE Online client behaves.
Complex software can fail in complex ways and client crashes not only take you out of the game unexpectedly, but can disrupt that critical moment you have been waiting for. In some cases, the cause is easy to find and a fix can be deployed quickly. In others, the crashes happen so rarely that it is very difficult to reproduce.
CCP takes client stability extremely seriously. Investments have recently been made to ensure our crash capturing ability is the best it can be, which should result in us finding more crashes and ultimately reducing the number you experience. This along with the bug reports you submit by using F12, will help us to keep the EVE client as stable as possible.
Upgraded reporting tech
When the client crashes, it will attempt to upload a crash report to a 24/7 crash monitoring system. After several years using the well-known and popular Breakpad library from Google, a better solution has been selected called the Crashpad library. Note that no information is sent to Google with either library.
Breakpad sits within the normal EVE client executable. This means that a client crashing can also cause Breakpad to crash, resulting in a lost report. The same situation also happens if the EVE Client runs out of memory; the valuable crash report may be lost.
Crashpad solved this problem by sitting outside the program it is monitoring. This means you will soon see a separate process running alongside the EVE Client appropriately named “eve_crashmon”.
It will look like this in Windows task manager:
This separate application uses no more memory or CPU than the old system it replaces so your experience should remain the same.
How crash reports are used
A crash report will typically fall into one of the five categories below: - The crash was caused by the client—confirmed with a reproduction. - The crash was likely caused by the client but with no confirmed reproduction. - The crash was caused by a device driver. - The crash was caused by external software interacting with the client. - The crash was likely caused by a hardware problem.
While it may sound counter-intuitive, having a crash confirmed as being caused by the client is the best-case scenario. There is a clearer path to implement a fix, test it, deploy it, and monitor the results.
If the client is likely at fault but the crash cannot be reproduced, your bug reports become increasingly invaluable – more on these shortly.
Driver issues are normally easy to spot as they will be from the same GPU manufacturer and often the same driver version number. Gathering helpful information from bug reports to give to the GPU manufacturer reduces their time to release a fix.
Software that interacts with the EVE client occasionally causes crashes. These are often game overlays, screen recording software, or hardware monitoring software that show statistics like GPU, CPU, FPS, and temperature information. Always make sure you're running the latest version of the software as well as the latest GPU driver.
Finally, hardware issues can be extremely frustrating as they vary from the client running flawlessly for hours to crashing on startup. They often have no clear pattern.
Bug reports are one of the most useful tools for solving issues in EVE Online. There are two ways to file a bug report and one is much better than the other. You can go to support.eveonline.com and press the yellow button (helpful) or you can press F12 in game (incredibly helpful).
If you think you have found an issue, please file a bug report even when you believe that someone else has already done so.
This is particularly important for technical issues where being able to cross-reference different systems gives much better visibility. For example, if a set of bug reports arrive where graphics are displaying incorrectly and all the reports mention the same GPU manufacturer and series of video card, testing then focuses on those cards to reproduce the problem in-house. This naturally leads to a faster fix being tested, deployed, and monitored.
The key to a great bug report is to include as much information as you can. A description of the issue, along with possible reproduction steps enables rapid replication.
When it comes to a client crash, it is extremely valuable to have a bug report filed if the client crashed because of a user action. For example, changing from borderless window mode to full screen causes a repeatable crash. Crash reports are not infallible and may be missing a critical piece of the puzzle that you can provide. Filing the report using the F12 in-game menu will include the important client logs about the crash too, even after a restart.
As part of CCP’s recent move into new offices, a new test lab was built to our specific requirements. This included increased available space, a more powerful air-conditioning system, and better storage for lots of testing hardware.
This is where fixes are tested before being deployed – especially when it comes to crash or graphical-related fixes. The lab tests against more than one CPU and GPU family, along with different versions of Windows and Mac operating systems.
The test lab is not just used for client bug fix testing, but also for performance and compatibility testing. Since EVE Online is run on so many different kinds of machines, this test lab checks that new releases work on the most popular configurations and that performance levels fall within expected values.
The test lab will continue to evolve in tandem with upgrades to the underlying code of the EVE Online client, including the move to DirectX 12 in the future.
Keeping your software up to date
Not every crash can be replicated in the test lab. Keeping your OS up-to-date is important not just for security, but also for ensuring that operating system defects get fixed. Both Windows and Mac will automatically update on a regular basis and players are encouraged to run the latest OS patches where possible.
Device drivers are a little more complex, as they sometimes require manually updating to the latest version. Reports frequently arrive from users that are running a graphics card driver that is several years old – and their issue is fixed by simply updating to the current driver.
One of the most difficult cases is the suspected hardware fault. Without having access to the physical machine, these are difficult to verify. Given enough time, every component in your computer will fail. When these failures occur, they normally show as a one-off crash report that has been caused by a situation that should be impossible for the client to find itself in.
While troubleshooting PC hardware issues can be complex, five of the more frequent issues with hardware related game crashing are: - GPU issues: Sometimes these will be shown as a corrupted image with unexpected colors, or a checker box effect being rendered. At other times, a game may drop back to the desktop with a “Display driver stopped responding and has recovered” message – although this is not always hardware related. - RAM issues: When RAM has failed, random crashing will often occur, or you will experience BSODs*. If you have a large amount of memory that has a single failed address, then it may only cause an issue very infrequently. - Overclocking: This has become more popular in the last few years due to motherboard manufacturers making it easier. While overclocking can offer rewarding performance increases, it does come with stability risks. If you encounter crashing issues with an overclock, it is recommended to return the system to stock values, and see if the issue persists. - PSU issues: Unfortunately, a bad power supply can make almost anything in your system look like it is at fault. It is recommended to use a quality power supply to ensure long term system stability. If a computer often crashes when under high load, then the power unit is something that should be investigated. - Overheating issues: These are usually caused by an excessive amount of dust being present or fans not sufficiently cooling due to bearing failure. Depending on what component is overheating, overheating can cause anything from poor performance to complete application crashing.
If you suspect you may have faulty hardware, thankfully there are many free programs to test your system stability. Popular recommendations are memtest86 for RAM and OCCT for general system stability. If you encounter any failures during system testing, the EVE client is likely to also experience problems at some point.
What does the future hold?
Our current crash reports include lots of useful information for debugging which significantly helps us resolve issues. However, they are unable to provide any context to wider issues such as how many players are experiencing this problem. CCP really wants to know “What percentage of players have experienced a new (previously unknown) crash so far today since the patch yesterday, and how does this compare to last week?” While we can find this information out with our current systems, it does involve the manual work of linking these events together.
In the future, we want dashboards showing us complex situations unfolding as they happen. To do this, we would need to monitor the health of all EVE Online clients in real-time. The move to Crashpad and having a crash monitoring process outside of the EVE Client is one of the first steps toward this.
Crashpad + Sentry as the endpoint for our crash reports allows us to monitor, filter, and take action quickly. Then to improve responsiveness, the next step is to integrate some of the recent improvements from their native SDK.
Thank you and remember to press F12 to file a bug report if needed.
CONCORD has been working on a major upgrade to the Encounter Surveillance System (ESS) and the new version is finally ready to roll out. The new ESS will come online alongside the Dynamic Bounty System (DBS) and together they will reshape ratting in Nullsec as you know it.
The original ESS was meant to offer Nullsec residents the chance to juice up their systems with better bounty payments at the risk of having of their earnings stolen by roaming player pirates. The concept was strong but clever players highlighted several cases - including risk/reward-related and deployment location issues - that exhibited much opportunity for improvement. With lots of learning now in-hand, the aim is to revisit the concept with more focus on creating a flashpoint for conflict across Nullsec.
- The ESS is no longer optional and will be present by default in all Nullsec solar systems.
- The location of the ESS is public and sits behind an acceleration gate with specific ship class access.
- Payouts from the ESS happen automatically to contributing PvE bounty earners and no longer need to be retrieved in person.
Breaking the Bank
Like the current ESS, a percentage of bounties generated in the solar system will be delivered to the ESS rather than the player's wallet. Unlike the current ESS, the take will now be split between two banks: - The Main Bank – This is where the majority of the money goes. It pays out automatically every 3 hours to contributors, but can be stolen from at any time by invaders. - The Reserve Bank – This is the real jackpot. A smaller portion of each bounty is placed in the Reserve Bank where it sits until claimed by someone on grid who has a consumable key required to unlock it. When stealing from the Reserve Bank, you will be paid over time at a rate that spools up to a maximum then winds back down. You will get to decide how long this cycle takes to complete, and in turn, how much you stand to gain. These keys will be available via gameplay content at a later date, and its availability, as well as the timing of its release, will depend on how the players will interact with the ESS after going live. More information to follow later!
For those wanting nitty-gritty details, you will have to wait until the feature is live and log in to discover them.
Earn Your Keep
The new ESS will have a unique set of on-grid rules to encourage good fights and avoid troll-like tactics. The ESS rule set, which affects a 75km radius around the structure, contains the following: - Warping is disabled - MJDs are disabled - MWDs are disabled - Cloaking is disabled - No cynos may be lit - No filaments may be activated
Together, these conditions will demand commitment from anyone hoping to get their hands on someone else’s hard-earned ISK. Of course, the meta that develops around the ESS will be monitored and adjustments will be made as needed.
All-in-all there is much excitement to see how this new version of the ESS pans out. More action for residents and roamers alike will emerge as players plan heists and protect their space.
Fly it like you stole it!
Champions of the Abyss has concluded and a huge congratulations to the overall winner - Cable Uta!
As announced back in July, to become one of the Champions of the Abyss capsuleers had to rack up the most wins cumulatively across all nine Proving Grounds events during Quadrant 3 - and many of you put in a lot of effort!
Here is the final Top 10 of the contest:
| Rank | Characer Name | Total Victories | | ---------- | ---------- | ---------- | | 1 | Cable Uta | 1678 | | 2 | uMillenium | 916 | | 3 | Tikktokk Tokkzikk | 746 | | 4 | Suitonia | 737 | | 5 | Gorski Car | 653 | | 6 | Frigate Gun | 627 | | 7 | SKYKILLER RUSSIAN | 586 | | 8 | NeedsToBeReal | 564 | | 9 | Kirzath | 557 | | 10 | Auraus Porcaleus | 507 |
The result should come as no surprise – Cable Uta finished in first place on the leaderboard of every single Proving Grounds event during the Zenith quadrant (even if sometimes just narrowly!) and ended up almost doubling the number of wins of his nearest rival, uMillenium.
The prizes for this contest have begun to make their way out to the winners.
The top 10 characters on the final Champions of the Abyss leaderboard will receive:
- 1000 PLEX
- A unique medal commemorating their achievement added to their in-game character sheet
The top 5 characters will also receive:
- a US$50 voucher and free postage for a single order from the EVE Online Gear Store
- Their names up in lights on an in-game billboard advertisement celebrating their prowess for all of New Eden to see
And finally, Cable Uta can probably expect the most prominent news outlet in the cluster to feature him an episode to air sometime soon!™
Thank you to all capsuleers who chased the glory of becoming one of the Champions of the Abyss and those of you who participated in the Proving Grounds over Quadrant 3 just for fun!
CONCORD is adopting a new Dynamic Bounty System (DBS) for bounty payouts across Nullsec space. This will reward those daring enough to hunt pirates in more dangerous space where their stronger presence is a greater threat to Capsuleer safety. Solar systems where pirate activity is under firm control by Capsuleers will see payments lowered. DBS is now live on the Singularity test server and will be live on Tranquility in November!
The universe will become both reactive to - and self-correcting from - Capsuleer behavior. These changes will not take the form of wild swings but will move slowly over the course of days to weeks in response to changes in activity.
This will see income generation more evenly spread across New Eden with large corporations needing to utilize more of their space over time if they want to keep their current levels of income. In addition, DBS will bring benefits to ongoing anti-botting efforts by making it easier to highlight suspicious activities and take appropriate actions.
How does it work?
Every solar system will now have an ever-changing bounty multiplier that is applied to any bounty payout earned in the system. As an example, if the solar system multiplier is at 110% and you kill a pirate that has 100,000 ISK bounty, the payout will now be 110,000 ISK. However, some of that will be captured by the revamped Encounter Surveillance System (ESS), which you will hear more about next week.
This multiplier will always be visible via the starmap so that you can locate high-value solar systems in which to hunt pirates.
This multiplier will get adjusted constantly based on what’s happening in the solar system:
- Excessive ratting? Multiplier goes down
- High level of player combat and death? Multiplier goes up
- Empty system? Multiplier stabilizes at an equilibrium value.
Of course there’s a lot going on behind the scenes to set the rate of change, but the three points above are all you need to effectively plan your bounty hunting activities.
The benefits of DBS
There are two main benefits that come from shifting to a more dynamic system:
- Dynamic systems and a constantly shifting ecosystem provide opportunities for dedicated Capsuleers to separate themselves from others. Learning to predict future high-pay areas, hunting in current low-pay areas (where you know there’s excessively safe ratting happening), and choosing your daily ratting grounds based on a new set of factors every day will make every part of Nullsec life more interesting.
- Spreading income generation across New Eden and moving away from massive ratting hubs supported by very concentrated infrastructure will create movement and more conflict. Empires will be stretched thinner and guaranteed protection will come at a more tangible cost.
This will ultimately determine how much space is necessary to support income generation based on individual player organization sizes and requirements.
Abundance breeds complacency and scarcity breeds war.
Predictable inputs lead to stagnant outputs.
Autarky is Anathema to Free Trade.
The DBS will begin impacting the playing field between small entities and large entities when it comes to the efficiency with which they can exploit their territory. This is a step to innovate and improve EVE Online for its third decade by introducing more dynamic systems that respond to player actions. The results will be unpredictable and truly rest in the hands of the capsuleers.
Rumor has it that CONCORD is nowhere near finished with its updates to bounty payouts in Nullsec space and is targeting the Encounter Surveillance System (ESS) next, so stay tuned for more on that next week!
The Monthly Economic Report for September 2020 is now available!
You can download all of the raw data used in this report here. As always, each image can be enlarged by clicking on it.
"A healthy economic environment where players can find opportunities by making interesting choices is the goal. One of the key pillars of EVE is that loss has meaning and a state is being reached where loss is not meaningful anymore for veteran players. It is imperative, both for EVE's success and for the well-being of its inhabitants, that the economy resides within a healthy state."
Back in March, the EVE Online Ecosystem Outlook blog described the plans for resource distribution in New Eden. This endeavour started last December with asteroid belt changes, then moved on to ore anomaly changes and a moon resource shake-up, and now the fourth step of the Shortage Phase is planned to go live in mid-October. Alongside the final steps of the Shortage Phase, the first step of the Redistribution Phase will also go live in the middle of October.
The main goal during the first steps of the Redistribution Phase is to create 'Primary Supply Zones' and give value to all space in New Eden. The process starts by creating the following map of minerals:
| Security Zone | Exclusive/Main Supplier of Mineral | Excluded Minerals | | ---------- | ---------- | ---------- | | Hisec | Tritanium | Isogen, Nocxium, Zydrine, Megacyte, Morphite | | Lowsec | Isogen, Nocxium | Tritanium, Zydrine, Megacyte, Morphite | | Nullsec | Zydrine, Megacyte, Morphite | Tritanium, Nocxium (some availability through Sov anomalies remains) | | Wormholes | - | Tritanium, Nocxium, Morphite |
This, as with anything else, will change according to how things in New Eden progress.
The above will be achieved through the changes detailed below.
Asteroid Ore DNA
The biggest change to come is with the ore DNA. This change is part of both the Shortage Phase and the Redistribution Phase. The Shortage part has to do with the exact values that were decided, while the Redistribution has to do with the material composition.
The mineral output of all asteroid ores will be changing in order to create room for a better geographical distribution of minerals, as stated above in the Redistribution Phase Section.
Here are the changes (only standard ores presented, appropriate changes are in place for higher yield ores):
| | Tritanium | Pyerite | Mexallon | Isogen | Nocxium | Zydrine | Megacyte | Morphite | | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | | Veldspar | 400 (-15) | | | | | | | | | Scordite | 150 (-196) | 90 (-83) | | | | | | | | Pyroxeres | 0 (-351) | 90 (+65) | 30 (-20) | | 0 (-5) | | | | | Plagioclase | 175 (+68) | 0 (-213) | 70 (-37) | | | | | | | Omber | 0 (-800) | 90 (-10) | | 75 (-10) | | | | | | Kernite | 0 (-134) | | 60 (-207) | 120 (-14) | | | | | | Jaspet | | | 150 (-200) | | 50 (-25) | 0 (-8) | | | | Hemorphite | 0 (-2200) | | | 240 (+140) | 90 (-30) | 0 (-15) | | | | Hedbergite | | 450 (-550) | | 0 (-200) | 120 (+20) | 0 (-19) | | | | Gneiss | | 2000 (-200) | 1500 (-900) | 800 (+500) | | | | | | Dark Ochre | 0 (-10000) | | 1360 (+1360) | 1200 (-400) | 320 (+200) | | | | | Crokite | 0 (-21000) | 800 (+800) | 2000 (+2000) | | 800 (+40) | 0 (-135) | | | | Bistot | | 3200 (-8800) | 1200 (+1200) | | | 160 (-290) | 0 (-100) | | | Arkonor | 0 (-22000) | 3200 (+3200) | 1200 (-1300) | | | | 120 (-200) | | | Mercoxit | | | | | | | | 140 (-160) | | Spodumain | 48000 (-8000) | 0 (-12050) | 0 (-2100) | 1000 (+550) | 160 (+160) | 80 (+80) | 40 (+40) | | | Bezdnacine | 40000 (+26800) | 0 (-2200) | 0 (-3780) | 4800 (+3700) | | | 128 (-57) | 0 (-10) | | Rakovene | 40000 (+37800) | 0 (-7200) | 0 (-855) | 3200 (+3200) | 0 (-450) | 200 (-70) | 0 (-83) | 0 (-2) | | Talassonite | 40000 (+27400) | | | | 960 (+504) | 0 (-81) | 32 (+19) | |
_Asteroid Belts _
- All variations of Omber and Kernite will be removed from Hisec asteroid belts.
- All variations of Veldspar, Scordite and Plagioclase will be removed from Lowsec asteroid belts.
- The quantity of all variations of Pyroxeres and Kernite will be reduced by 75%.
- The quantity of all variations of Hemorphite and Hedbergite will be increased by 400%.
- All variations of Scordite, Plagioclase, Omber, Jaspet, Hemorphite, Hedbergite, Gneiss, Dark Ochre, and Crokite will be removed from Nullsec asteroid belts.
- The quantity of all variations of Kernite will be reduced by 75%.
- The quantity of all variations of Bistot will be reduced by 70%.
- The quantity of all variations of Arkonor will be reduced by 50%.
- The quantity of all variations of Mercoxit will be reduced by 90%.
- All Ore Anomalies will be removed from Hisec systems.
- Certain Ore Anomalies will be removed from Lowsec systems.
- The quantity of all variations of Gneiss and Dark Ochre in all Lowsec Ore Anomalies will be increased by 300%.
- The quantity of all variations of Crokite in all Lowsec Ore Anomalies will be increased by 9900%.
- The following ores and their variations will be removed from Wormhole Anomalies:
- Veldspar, Scordite, Plagioclase, Jaspet, Hemorphite, Hedbergite, Dark Ochre, Crokite, Mercoxit, Spodumain.
Sov Ore Anomalies
The ore quantity composition of the Sov Ore Anomalies will be as follows:
| Level | Pyroxeres | Omber | Kernite | Crokite | Bistot | Arkonor | Mercoxit | | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | | 1 | 600,000 | | | | 5,000 | 5,000 | | | 2 | 200,000 | 10,000 | | | 40,000 | 25,000 | 1,500 | | 3 | | 20,000 | | | 60,000 | 50,000 | 2,000 | | 4 | | | 20,000 | 2,500 | 135,000 | 50,000 | 2,500 | | 5 | | | | 5,000 | 175,000 | 50,000 | 3,000 |
Further changes are being made to other resources:
K-Space Gas Sites
- The spawn probability of Mykoserocyn sites will be reduced.
- The spawn probability of Cytoserocin sites will be reduced.
W-Space Gas Sites
- The respawn time for all Fullerite gas sites will be increased.
- The spawn probability of all Fullerite gas sites will be reduced.
- All Fullerite quantities at all sites will increase by 100%.
- Certain Fullerite sites will be removed from higher class wormholes.
- The spawn probability of all Ice Sites will decrease.
As mentioned in the EVE Online Ecosystem Outlook blog from March, the goal for this update is the reduction of inventory. Alongside that, the following is expected: - More movement in Lowsec - and potentially more destruction - Increase of mining ship losses - More market transactions for ores/minerals - Mineral income from refining items to increase - Prices of minerals to change
This has now been almost a year long journey, and there is a lot more to come. For some this has been painful, but significant progress is being made towards the goals for a healthy New Eden ecosystem as mentioned in the EVE Online Ecosystem Outlook blog.
On this journey there have been very encouraging signs, and as New Eden reaches its final stages of the Shortage Phase, the journey into the Redistribution Phase begins.
The New Eden Ecosystem, Resource Distribution, and more will be discussed on Friday 25 September 18:00 EVE time with CCP Rattati, CCP Psych, Carneros & Caleb Ayrania. Make sure you catch it live on the CCP Twitch channel!
The Monthly Economic Report for August 2020 is now available!
You can download all of the raw data used in this report here. As always, each image can be enlarged by clicking on it.
It’s time to level up and push your Project Discovery performance to an even higher benchmark, as the data coming in will now be more complex, resulting in even better learnings for the science community! As immortal pioneers of New Eden, you have surpassed even the highest expectations for this project already, and those were expectations earned through your stellar performances in previous incarnations of Project Discovery.
“We have been working on Project Discovery for 5 years and it is amazing to see how the EVE player community even after all this time keeps tirelessly participating in research efforts. This contribution is especially important and valuable in these pressing times. Thank you all!” - MMOS CEO and co-founder Attila Szantner.
In designing and implementing this latest version of Project Discovery, a lot has been learned about flow cytometry and cell populations to make the content engaging while also delivering accurate results to our scientists. Given that 39.6 million submissions have been amassed since the project’s launch, it’s safe to say that EVE Online’s players have answered the call to arms in the fight against COVID-19.
“The participation and quality of the data produced by the EVE community has been truly amazing and exceeded all our projections. We are thrilled to be able to mine this unique source of wisdom for the immediate benefit of biomedical research” - Jérôme Waldispühl, Associate Professor, School of Computer Science, McGill University.
With 466,000 already verified and now able to be used in scientific research, it’s also fair to say that this is a quantity and accuracy of data that would not exist without the efforts of EVE’s dedicated players around the world, but with so many samples still flooding in, it’s time to level up the accuracy.
When the project was launched, the main aim was to teach you how to demarcate unique populations in flow cytometry graphs while sending the end result of your efforts, the analyzed graphs, to CCP’s partners at UNIMORE, MMOS, McGill, and the University of British Columbia who would then use that data to understand COVID-19 better. Solid progress has been made on that front, which is fantastic, and if you’d like to learn more about that, then give this podcast a listen or visit McGill’s website about the project. That said, it was always known that at some point in time the approach would need to be refined, and it turns out that that time is now.
“EVE Online players are doing an exceptional job. The accuracy of their analyses is very high, which demonstrates the extraordinary attention that is paid to what is actually not just a game but a real fight against COVID. Players from all over the world are proving that collaboration with scientists is absolutely successful and that together we can do a lot, in a very difficult moment in which the uncertainties are much greater than ever.” - Dr. Andrea Cossarizza, Professor of Pathology and Immunology at the University of Modena and Reggio Emilia School of Medicine in Italy.
While Project Discovery participants have produced invaluable flow cytometry data over the past three months, after our scientific partners compiled your submissions and looked them over with their decades of laboratory experience, they have now issued a challenge for you to produce even more accurate graphs. To do that, though, you will need to learn a few additional things about flow cytometry and the plots being used, as well as what could be done better.
For most of the Capsuleer-analyzed plots, the data being produced is the gold standard of gold standards. Anything that looks like the following graphs is already being analyzed adeptly. Don’t change your approach for these.
The hope is to improve clustering in charts that look like these two, which are more difficult to properly analyze because of the peculiarities involved in turning real-world blood and cell samples into 2D plots.
It’s best to start by talking about what makes Type A samples unique: their fancy tails. Known as ‘smears’ in the world of flow cytometry, the thin band of cells on either side of the high-density cluster contain valuable information about how COVID-19 impacts cells. The presence of smears mean that cells are maturing slowly and that there are several cells in-between new and mature. In short, there are a lot of cell types present even though there aren’t that many dots on the plot. Figuring out where the populations are, that is, where the ‘clusters’ of cells are, is incredibly valuable when trying to understand viral impact on cell populations. These kinds of plots are a big deal and while it does not appear that there is a lot of information on them about cells, there definitely is.
Type A samples are also tricky to interpret because of a design choice made at CCP in tandem with MMOS and the scientists, namely, to remove the interval marks on either side of the plot. The removal of the labeling of these graphs was done in part to combat botting and in part to make a more streamlined user experience. That included removing the tick marks, which matters because all the plots we’ve been given are mapped on logarithmic axes. This means that as the number on the interval mark gets bigger, there are exponentially more proteins present at that portion of the cell. One dot becomes 1000 proteins at the tick mark labeled “3” on the Type A sample. Essentially, one dot at the top right of the plot is equivalent in protein amount to thousands of dots in the bottom left of the plot.
So how does this impact Capsuleer clustering? Simply stated, it is important to make sure to separate smears from high-density clusters. Instead of producing charts that look like this…
…we need to instead produce charts that look like this:
Doing this is relatively easy on plots where the side smear aligns with the high-density cluster like above. In this case, just demarcate the bottom smear by aligning your polygon with the lower end of the opposite smear. Try to mirror your clusters. Where it gets to be a bit more complicated is in samples like this:
In this sample, the side smear to the left is much smaller than the other and would cut straight through the high-density cluster if mirrored. Don’t do this. Instead, think of these samples like comets, where you’re trying to separate the tail of the comet from its body. Doing so will produce even more valuable data about these particularly unique cell populations.
Comet plots aside, it is also important to address the Type B graphs and how those can be better clustered. As seen above, these cytometry plots look like an amalgamation of cell populations. Until now, Capsuleers have done an excellent job of demarcating the left, right, and middle of Type B charts. However, you’ll notice that there is a cluster of data points sort of sliding off of the high-density area. Think of these as rogue cells, vying to break free from the larger cluster. They need to be categorized as a separate group accordingly.
To do so, one must simply mark the polygon at the thinnest point of separation between the two clusters. When that cannot be easily determined, try to mirror each cluster off the other and draw a separating line through the middle of the mirror.
All of this said, it is beyond impressive how many robust samples EVE Online players have submitted thus far, and there is no doubt at all that you’re up for the challenge of more accurately demarcating Project Discovery’s trickiest datasets yet.
“This project is crashing through all my expectations, with players continuing to show great engagement and interest in the work we are doing, as well as providing huge amounts of high-quality data for our research. Their efforts will not only contribute to the understanding of COVID-19, but the data they are generating will also be freely and widely shared with the entire scientific community. There is very high interest in re-using their results for the generation of machine learning algorithms. There is simply no other resource out there for this anywhere close to what is now being generated.” - Dr. Ryan Brinkman, Professor in Medical Genetics, the University of British Columbia, Distinguished Scientist at BC Cancer.