Sunday, January 27, 2019

Using small details to add additional context to other artifacts

This is a quick post on some files that could add additional context to other artifacts.

VLC media player (tested version: 3.0.5 / 3.0.6)
Artifact: "restart the playback where left off"
So what?: instead of simply saying that a file was opened with VLC, it could be possible to prove that a media file was also watched from beginning to a certain (milli)second.
Description: if a media file is partially played, VLC tracks the last played position to allow the user to resume playback when reopening the same file. Depending on the operating system, the last played position value is stored in the following files:
  • Windows: C:/Users/<username>/AppData/Roaming/vlc/vlc-qt-interface.ini
  • Ubuntu: /home/<username>/.config/vlc/vlc-qt-interface.conf
  • macOS : /Users/<username>/Library/Preferences/org.videolan.vlc.plist
VLC for Windows and Ubuntu stores the values in the [RecentsMRL] section within the vlc-qt-interface file and these values are expressed in milliseconds. VLC for macOS stores the values in the recentlyPlayedMedia array within a .plist file and the values are expressed in seconds. Based on my tests, a zero value may either mean that a media file has been fully played or that less than five percent of the file contents has been played. 

To automate the parsing I wrote a Python 2.7 script. Here's an example of the output of the script VLC_LastPlayedPosition.py:

C:\>VLC_LastPlayedPosition.py vlc-qt-interface.ini

Analyzing file: C:\temp\vlc-qt-interface.ini...

------------------------------------------
 VLC media player ('RecentsMRL' section)
 The entries are listed by default from
 the most recent to the oldest
------------------------------------------

# | Last Played Position (h:mm:ss) | Media file
1 | 0:04:05 | file:///C:/DATI/audio-video/file2.mp4
2 | 1:26:16 | file:///C:/DATI/audio-video/file1.avi

Output saved to: 20190127_150900_vlc.csv


Adblock Plus (3.4.2) add-on for Firefox (64.0.2)
Artifact: websites that have been whitelisted by the user
So what?: the user visited a website that required ad blockers to be turned off. The user had to manually disable Adblock Plus for that website.
Description: Adblock Plus is a popular ad blocker. The user can choose to allow a site to show ads by "whitelisting" it. When that happens, an entry is added to a storage.js file which is located at:
  • Windows: C:\Users\<username>\AppData\Roaming\Mozilla\Firefox\Profiles\<profileID>.default\browser-extension-data\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}\storage.js
  • Ubuntu: /home/<username>/.mozilla/firefox/<profileID>.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js
  • macOS: /Users/<username>/Library/Application Support/Firefox/Profiles/<profileID>.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js
On a live machine, the list of all the websites that have been whitelisted can be viewed from Firefox by going to:
  • about:addons | Adblock Plus options | Whitelisted websites tab
That list can be directly extracted from the file mentioned above by using the following regular expression:
  • \[Subscription\]","url=~user~\d*","defaults=whitelist","","\[Subscription filters\]
The entries we're looking for are listed right after the string matching the regular expression. 
This is my script to automate the parsing: Firefox_AdblockPlus.py. Here's an example of the output:

C:\>Firefox_AdblockPlus.py --default-path

# Analysis of Adblock Plus for Firefox #
File: C:/Users/xxxx/AppData/Roaming/Mozilla/Firefox/Profiles/
xxxx.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js

Whitelisted websites added by user: 2
- macworld.com
- windowscentral.com

I know that this add-on is also available for Chrome, but Chrome uses LevelDBs instead of the storage.js file and at the moment I don't know how to extract what I need in a reliable way.


NoScript (10.2.1) add-on for Firefox (64.0.2)
Artifact: web sites that have been manually set to "trusted" or "untrusted" by the user.
So what?: The user visited a website and, within the browser, manually trusted or untrusted the domain by changing the default settings through the NoScript icon.
Description: NoScript is a security add-on. When the user sets a domain to "trusted" or "untrusted", an entry containing the domain (not the full URL) is added to a .sqlite database named storage-sync.sqlite:
  • Windows: C:/Users/<username>/AppData/Roaming/Mozilla/Firefox/Profiles/<profileID>.default/storage-sync.sqlite
  • Ubuntu: /home/<username>/.mozilla/firefox/<profileID>.default/storage-sync.sqlite
  • macOS: /Users/<username>/Library/Application Support/Firefox/Profiles<profileID>.default/storage-sync.sqlite
From Firefox, NoScript settings can be reviewed by going to:
  • about:addons | NoScript options | Per-site Permissions tab
The same data can be retrieved by directly analyzing the collection_data table within the storage-sync.sqlite file. The record having collection_name = default/{73a6fe31-595d-460b-a920-fcc0f8843232} and record_id = key-policy contains the entries we're looking for. The entries have no timestamp and are stored in JSON format in the record field in the sites object. Some domains are already in the file as part of NoScript default settings.

For instance, I navigated to https://www.cnn.com and set some domains to "trusted" and some others to "untrusted".


From the list alone extracted from the .sqlite file, we don't know if the entries refer to domains visited by the user or domains simply "trusted/untrusted" by the user when visiting something else.


After several trial and error attempts, I've noticed it's possible to distinguish between "visited sites" and "sites not visited" by using HTTP requests and observing the responses. It's possible that the user has visited a domain if the domain:
  • returns a HTTP response
  • doesn't redirect to another domain
  • doesn't have a very small Content-Length size
Moreover, if a domain loads scripts residing on another domain, it's highly probable that the former is the domain that was directly visited by the user.

If all or some of the above conditions are not met, then it's possible that the user has set a domain to "trusted" or "untrusted" when visiting any of the visited domains.

The script I've developed can be downloaded here: Firefox_NoScript.py.
If the script is used with a "-r" option, it will send a HTTP request to each site found in the file and try to separate the results. The more entries there are in the .sqlite file, the more accurate the script output is.


C:\>Firefox_NoScript.py --default-path -r

Firefox_NoScript v.20190127
Script to extract the permissions that have been manually added to NoScript add-on

Analyzing file: C:/Users/xxxx/AppData/Roaming/Mozilla/Firefox/Profiles/
xxxx.default/storage-sync.sqlite ...

Non-default permissions found: (6)

Sending HTTP requests to 6 domains found in the file...

   Based on the HTTP responses received, it's possible that:
     ==> the user directly visited 1 domain(s):
      - cnn.com

     ==> the trust level for 5 domain(s) was set by the user
     when visiting other domains:
      - chartbeat.com
      - cookielaw.org
      - optimizely.com
      - postrelease.com
      - sharethrough.com

Output saved to: 20190127_141136_NoScript.csv

The Tor Browser (8.0.4) comes with NoScript (10.2.1) pre-installed. The .sqlite file is located at:
  • C:\Users\<username>\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default\storage-sync.sqlite
Differently from Firefox, and based on my tests, the storage-sync.sqlite file will retain NoScript last session's settings when the Tor Browser is closed. The settings are lost when the Tor Browser is reopened.


4 comments:

  1. Yo, G...pretty fascinating stuff! Thanks for sharing!

    ReplyDelete
  2. for the vlc stuff, did a quick test on my mac. Can confirm that playback is in seconds

    "recentlyPlayedMedia" => {
    "file:///Users/___/Music/This Month in 4n6/ThisMonthIn4n6-01_19.mp3" => 361

    ReplyDelete
    Replies
    1. I tested my script on macOS Sierra (10.12) with VLC media player 3.0.6 installed. Thanks Phill for confirming my findings. I wonder why VLC doesn't use milliseconds on macOS.

      Delete