This repository was archived by the owner on Sep 19, 2020. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 19, 2020. It is now read-only.
All pages take forever to load #269
Description
Prerequisites
- I performed a cursory search of the issue tracker to avoid opening a duplicate issue
- Your issue may already be reported.
- I also searched the existing issues at https://github.com/gorhill/uMatrix/issues
- This is not a support issue or a question
- Support issues and questions are handled at /r/uMatrix
- I tried to reproduce the issue when...
- uMatrix is the only extension
- uMatrix with default lists/settings
- using a new, unmodified browser profile
- I am running the latest version of uMatrix
- I checked the documentation to understand that the issue I report is not a normal behavior
- I used the logger to rule out that the issue is caused by my ruleset
Description
After Firefox's storage becomes damaged, uMatrix causes all pages to take forever to load.
If Firefox is to blame, I would still suggest to handle the error gracefully, rather then just silently blocking all pages from being loaded (which renders entire Firefox unusable until problem is fixed). Probably show a reasonable error message to help user get over it?
The problem has happened to me 3 times now: today, a couple weeks ago and around a year ago.
I understand that other users faced this too, for example #264 - and I think I saw others, but can't find them now.
A specific URL where the issue occurs
Any url.
Steps to Reproduce
- Close your Firefox.
- Go to your Firefox
<profile_name>\storage\default. For example, on Windows:%appdata%\Mozilla\Firefox\Profiles\<profile_name>\storage\default - Backup, then delete all files in this folder. Can't say why this is important.
- Extract attached archive: 7.zip. I understand that it contains a damaged storage entry. Can't say what exactly is wrong about it.
- Start Firefox, try to navigate to any URL. Firefox will take forever to load the page.
Ruleset
Unrelated to ruleset.
Supporting evidence
Tools | Web Developer | Browser Console will have this:
20:22:58.060 IndexedDB UnknownErr: ActorsParent.cpp:570
20:22:58.064 UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code. ExtensionStorageIDB.jsm:831
normalizeStorageError resource://gre/modules/ExtensionStorageIDB.jsm:831
method chrome://extensions/content/child/ext-storage.js:273
AsyncFunctionThrow self-hosted:697
20:22:58.168 IndexedDB UnknownErr: ActorsParent.cpp:570 2
20:22:58.274 UnknownError: The operation failed for reasons unrelated to the database itself and not covered by any other error code. ExtensionStorageIDB.jsm:831
normalizeStorageError resource://gre/modules/ExtensionStorageIDB.jsm:831
method chrome://extensions/content/child/ext-storage.js:273
AsyncFunctionThrow self-hosted:697
20:22:58.274 µm.userSettings is undefined storage.js:136
settingsLoaded moz-extension://d7631edc-38b5-4fa6-85c4-c4a8d89027b6/js/storage.js:136
apply self-hosted:1870
applySafeWithoutClone resource://gre/modules/ExtensionCommon.jsm:614
wrapPromise resource://gre/modules/ExtensionCommon.jsm:843
withLastError resource://gre/modules/ExtensionCommon.jsm:762
wrapPromise resource://gre/modules/ExtensionCommon.jsm:831
Your environment
- uMatrix version: 1.4.0
- Browser Name and version: Firefox 77.0.1
- Operating System and version: Win8.1
Activity
gorhill commentedon Jun 13, 2020
Please see if you can also reproduce with dev build: https://github.com/gorhill/uMatrix/releases/tag/1.4.1b6
homo-programmatis commentedon Jun 13, 2020
Yes, reproducible with 1.4.1.6.
In case you didn't notice: I have provided reproduction steps. I have verified that they work when starting with a clean profile.
gorhill commentedon Jun 13, 2020
Can you share why you felt the need to say this?
homo-programmatis commentedon Jun 13, 2020
Totally no offense meant. The report page (especially with header) is quite long. In other couple reports I glanced at, reproduction steps are not really reproduction steps. So I imagined that it's easy enough to not notice that this time, reproduction steps actually do reproduce.
uBlock-user commentedon Jun 13, 2020
It's not just an error, storage component is broken and that breaks uMatrix along the way along with all extensions too relying on that component.
gorhill commentedon Jun 13, 2020
I would still like to see if the start code can better deal with such error -- now that I have a test case I will be able to step in the code as the error occurs.
homo-programmatis commentedon Jun 13, 2020
Let me try to provide some reasoning.
Such cases inevitably fall in some border zone, where some other component (Firefox, in this case) can be blamed and that's it, discussion over. I agree, my naive understanding is that Firefox should have dealt with the problem without letting it cause more damage.
However, let's see it with the eyes of an average user:
succeedmr123 commentedon Jun 16, 2020
Dear and appreciated @gorhill & @homo-programmatis who disclosed this very unpleasant issue first,
I can fully second the observations stated here, all points 1-6 nails the user choices. It all happened to me with the another addon. Addon data is still inaccessible in its own IndexDB, addon does not function at all (as it can not acces data), and it happened now with uMatrix again.
I will try to get to the suspected culprit as short as possible, and propose a minimal FF auto-check fix mechanism;
In my opinion, we have a case of Firefox not having a resilient mechanism of protecting IndexDB data structures, these are confirmed by my investigation at least. Simply, the data in an IndexDB are hanging there to be corrupted at the mercy of a sudden power loss or any similar disaster, without autofix chackup.
Last time it happened (and is not addressed asof yet at all) was the addon called "Tab Session Manager".
So, suspected cause of this is similar to IndexDB corruption.This time did not have the moment yet (as I deal with 16 hours production time here). Or, from my surface user level observation, a general issue of unfixed.. any kind of a setup, database, config file, if a sudden power loss occurs.
I found myself dumbfounded with Firefox/addon weird behaviour now for the second time - this time with uMatrix. However, it seems it is not uMatrix direct "fault". Firefox local data structure for the addon is, I speculate, broken, and any pages totally refuse to load due to uMatrix not being able to load its data, and then not letting pages get the datastream. Under uMatrix settings, I see 0 bytes in local storage, talking from an user superficial overview.
Given my experience with the first addon with broken IndexDB "Tab Session Manager", something like that is the culprit here as well.
Firefox, as any complex system of OS calibre of complexity, should be having a mechanism to autodetect dirty data and before start - autorepair it, either from backup, or live.
Can not be normal that the users get into a situation of more than one addon get Firefox stalled, or not functioning, because the addon can not load data that is not sanitized properly.
And users themselves not being able to do anything about it.
For instance, how I found at all that IndexDB was at fault;
was loking for IndexDB fixer out at the internet - no dice.
So, average user, even the one who have the best willpower to leave production machine workflow and investigate, can not do it without the platform mechanism /fix for similar data corruption scenarios
Now, I have another situation with a broken addon, and its uMatrix. It's off for the third day (icon itself is all greyed out, not functioning), and I am forced to be running sites with privacy concerns, my (i.e. user's) data up to the date of breakage is inaccessible yet, without any way to fix the situation for now. Just a first-hand user experience.
I am almost ready to jump in fire and start debugging hard-core out of sheer feeling of frustration at the inadequacy of the current FF robustness, either desktop release, or Preview Edition for Android, who started to crash 90% of the time.
Best,
succeedmr123 commentedon Jun 18, 2020
Dear respected @gorhill and @homo-programmatis
two additional QA tests are documented here in an effort to exclude the culprits til eventually pinpointing the exact factors contributing to the issue;
First crude / fast glance checkup reveals this:
(TL/DR, different machines work differently it seems windows 10 v1909 64 bit has to do with it in some way according to my preliminary tests)->
Platform Test: uMatrix on few other computers - different systems, same Firefox account, logged in, and syncing data.
Functional test - lets get to results:
PC1: Win 10 v1909 64 bit, Firefox 64 bit of course, synced, uMatrix addon on:
Result: unusable, traffic blocked, click on the grey uMatrix icon gives No web page, needed to shut the addon off.
PC2 Win 10 v1909 64 bit - also unusable, addon also need to be shut off in order for Firefox to load any page at all
PC3 Win7 32 bit, Firefox ESR 32 bit - it works well
Hope the obvious results may help.
Test 2: Test by disabling ASLR anti-exploit under Windows defender options per-app for firefox as exclusion (TL/DR, attempt 1, test did not did not make any difference)
Turned off Antivir exploit "obvious suspect" - ASLR page anti-exploit technique 1 and 2 (there are two methods near the end of the list within windows defender)
Later will proceed with excluding Firefox for all security anti-hack methods stated there in order to observe the behaviour (We have known confirmed test that e.g. Jetbrains IDE get totally messed up by Defender anti-exploit tech)
Hope that helps
homo-programmatis commentedon Jun 18, 2020
This is how to fix and find which folder is causing the problem:
<profile_name>\storage\default. For example, on Windows:%appdata%\Mozilla\Firefox\Profiles\<profile_name>\storage\defaulthttps+++.succeedmr123 commentedon Jun 21, 2020
@homo-programmatis
I appreciate the basic logic flow outline, however..
Reasons not to do that way, the alternative, and a long-term fix:
ALTERNATIVE to fix the unacceptably long time expense - its way more easy to find addon ID, then matching with addon Folder with the similar ID, and if anyone is into destroying his own data, click delete and reinstall the addon
Long term fix: autocheck at addon, or Firefox level, as a system to autofix the misbehaving databases, and anything inbetween. Think something (metaphorically) along the lines of M$ Chkdisk, fixboot,
rossengeorgiev commentedon Jun 22, 2020
Hi, I've been encountering this issue since March. Basically, storage breaks somehow when a site uses it. Say, youtube does it. Open YouTube, close browser, re-open, nothing loads until site folders are deleted from
storage/defaultfolder. Here are the errors from the console:I not able to replicate the behaviour on another PC. I wonder if there is feature flag, or settings, in the browser that could be causing the different behaviour?
rossengeorgiev commentedon Jun 22, 2020
I have no trailing dot, and the paths are <130 chars. Toggling the suggested settings had no effect.
SyntevoAlex commentedon Jun 22, 2020
If you have some patience, please try my algorithm from #269 (comment)
If you already suspect that youtube causes it, could you please backup&remove all other folders to verify that just the youtube's folder is enough?
rossengeorgiev commentedon Jun 22, 2020
hi @SyntevoAlex, here is a video to illustrate. This is the minimum repro case I have https://streamable.com/voi74a This is with umatrix disabled, so looks like Firefox bug.
Raised the issue here: https://bugzilla.mozilla.org/show_bug.cgi?id=1647323
homo-programmatis commentedon Jun 24, 2020
Thanks for the video. Your case looks pretty bad indeed. You did the right thing reporting it to Mozilla directly. In my case, the issue is not easily reproducible.