Showing posts with label Computer Forensics. Show all posts
Showing posts with label Computer Forensics. Show all posts

Friday, December 4, 2020

Let's combine EvtxEcmd with LogonTracer

This blog post aims to show how to combine EvtxECmd (v0.6.0.3) with LogonTracer (v1.5.0) during the analysis of Windows Event Log events.

To have some data to work on, download for instance the Windows EVTX Samples shared by Samir @SBousseaden on GitHub.

Extract the EVTX files to a temporary location like C:\TEMP\EVTX-ATTACK-SAMPLES-master.

Use EvtxECmd to extract only the events that are currently supported by LogonTracer: this is especially useful when dealing with cases that contain GBs of EVTX files from different systems and you want to load into LogonTracer just the event IDs that this tool can interpret, speeding up the whole process.

EvtxECmd.exe -d <PATH> --xml <PATH> --xmlf evtxecmd.xml --inc 4624,4625,4768,4769,4776,4672

Now we need to install LogonTracer. For convenience, I'll install the docker version of the tool on a REMnux v7 VM which runs Ubuntu 20.04. You may need to increase the amount of memory assigned to the VM (at least 4 GB).

sudo apt-get update && sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common && curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - && sudo apt-key fingerprint 0EBFCD88 && sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" && sudo apt-get update && sudo apt-get install docker-ce docker-ce-cli containerd.io && sudo docker pull jpcertcc/docker-logontracer && sudo usermod -aG docker remnux

From the terminal in REMnux, launch LogonTracer:

docker run --detach --publish=7474:7474 --publish=7687:7687 --publish=8080:8080 -e LTHOSTNAME=127.0.0.1 jpcertcc/docker-logontracer

Open the browser on REMnux and go to: http://127.0.0.1:8080/

Scroll down at the very bottom, till you see on the left the button "Upload Event Log".
Click on it and: 
  • select "XML" in the next window that appears
  • set the time zone
  • choose the evtxecmd.xml file that we created before. 
Once you click "Upload", the parsing fails due to an error saying "This file is not XML format".


The XML structure of evtxecmd.xml has to be tweaked a little in order to work with LogonTracer. After comparing it with an XML file created using Microsoft Event Viewer, and based on my tests, in order to make the file work you need to:
  • Add the following header: <?xml version="1.0" encoding="utf-8" standalone="yes"?><Events>
  • Add this footer: </Events>
  • Replace all the <Event> tags with <Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>
  • Merge all lines into one
  • Remove all the spaces between ">" and "<"

We can use PowerShell to automate the above steps. 

$evtxecmd = "C:\TEMP\EVTX-ATTACK-SAMPLES-master\evtxecmd.xml";$logontracer = '<?xml version="1.0" encoding="utf-8" standalone="yes"?><Events>' + [System.IO.File]::ReadLines($evtxecmd) + "</Events>";$logontracer = ($logontracer -join ("")) -replace "<Event>","<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" -replace ">[ ]*<","><";$logontracer | Out-File logontracer.xml -Encoding utf8

Let's try again with LogonTracer and this time the parsing succeeds!



[UPDATE 2020-12-07]: Just published a script (ConvertTo-LogonTracer.ps1) that can do the conversion, searches for keywords and splits a large input file to smaller chunks. Edit the settings inside the script before running it. Give it a try! The script is on my GitHub repo: https://github.com/forensenellanebbia/powershell-scripts/blob/master/ConvertTo-LogonTracer.ps1


References

Sunday, January 27, 2019

Using small details to add additional context to other artifacts

This is a quick post on some files that could add additional context to other artifacts.

VLC media player (tested version: 3.0.5 / 3.0.6)
Artifact: "restart the playback where left off"
So what?: instead of simply saying that a file was opened with VLC, it could be possible to prove that a media file was also watched from beginning to a certain (milli)second.
Description: if a media file is partially played, VLC tracks the last played position to allow the user to resume playback when reopening the same file. Depending on the operating system, the last played position value is stored in the following files:
  • Windows: C:/Users/<username>/AppData/Roaming/vlc/vlc-qt-interface.ini
  • Ubuntu: /home/<username>/.config/vlc/vlc-qt-interface.conf
  • macOS : /Users/<username>/Library/Preferences/org.videolan.vlc.plist
VLC for Windows and Ubuntu stores the values in the [RecentsMRL] section within the vlc-qt-interface file and these values are expressed in milliseconds. VLC for macOS stores the values in the recentlyPlayedMedia array within a .plist file and the values are expressed in seconds. Based on my tests, a zero value may either mean that a media file has been fully played or that less than five percent of the file contents has been played. 

To automate the parsing I wrote a Python 2.7 script. Here's an example of the output of the script VLC_LastPlayedPosition.py:

C:\>VLC_LastPlayedPosition.py vlc-qt-interface.ini

Analyzing file: C:\temp\vlc-qt-interface.ini...

------------------------------------------
 VLC media player ('RecentsMRL' section)
 The entries are listed by default from
 the most recent to the oldest
------------------------------------------

# | Last Played Position (h:mm:ss) | Media file
1 | 0:04:05 | file:///C:/DATI/audio-video/file2.mp4
2 | 1:26:16 | file:///C:/DATI/audio-video/file1.avi

Output saved to: 20190127_150900_vlc.csv


Adblock Plus (3.4.2) add-on for Firefox (64.0.2)
Artifact: websites that have been whitelisted by the user
So what?: the user visited a website that required ad blockers to be turned off. The user had to manually disable Adblock Plus for that website.
Description: Adblock Plus is a popular ad blocker. The user can choose to allow a site to show ads by "whitelisting" it. When that happens, an entry is added to a storage.js file which is located at:
  • Windows: C:\Users\<username>\AppData\Roaming\Mozilla\Firefox\Profiles\<profileID>.default\browser-extension-data\{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}\storage.js
  • Ubuntu: /home/<username>/.mozilla/firefox/<profileID>.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js
  • macOS: /Users/<username>/Library/Application Support/Firefox/Profiles/<profileID>.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js
On a live machine, the list of all the websites that have been whitelisted can be viewed from Firefox by going to:
  • about:addons | Adblock Plus options | Whitelisted websites tab
That list can be directly extracted from the file mentioned above by using the following regular expression:
  • \[Subscription\]","url=~user~\d*","defaults=whitelist","","\[Subscription filters\]
The entries we're looking for are listed right after the string matching the regular expression. 
This is my script to automate the parsing: Firefox_AdblockPlus.py. Here's an example of the output:

C:\>Firefox_AdblockPlus.py --default-path

# Analysis of Adblock Plus for Firefox #
File: C:/Users/xxxx/AppData/Roaming/Mozilla/Firefox/Profiles/
xxxx.default/browser-extension-data/{d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d}/storage.js

Whitelisted websites added by user: 2
- macworld.com
- windowscentral.com

I know that this add-on is also available for Chrome, but Chrome uses LevelDBs instead of the storage.js file and at the moment I don't know how to extract what I need in a reliable way.


NoScript (10.2.1) add-on for Firefox (64.0.2)
Artifact: web sites that have been manually set to "trusted" or "untrusted" by the user.
So what?: The user visited a website and, within the browser, manually trusted or untrusted the domain by changing the default settings through the NoScript icon.
Description: NoScript is a security add-on. When the user sets a domain to "trusted" or "untrusted", an entry containing the domain (not the full URL) is added to a .sqlite database named storage-sync.sqlite:
  • Windows: C:/Users/<username>/AppData/Roaming/Mozilla/Firefox/Profiles/<profileID>.default/storage-sync.sqlite
  • Ubuntu: /home/<username>/.mozilla/firefox/<profileID>.default/storage-sync.sqlite
  • macOS: /Users/<username>/Library/Application Support/Firefox/Profiles<profileID>.default/storage-sync.sqlite
From Firefox, NoScript settings can be reviewed by going to:
  • about:addons | NoScript options | Per-site Permissions tab
The same data can be retrieved by directly analyzing the collection_data table within the storage-sync.sqlite file. The record having collection_name = default/{73a6fe31-595d-460b-a920-fcc0f8843232} and record_id = key-policy contains the entries we're looking for. The entries have no timestamp and are stored in JSON format in the record field in the sites object. Some domains are already in the file as part of NoScript default settings.

For instance, I navigated to https://www.cnn.com and set some domains to "trusted" and some others to "untrusted".


From the list alone extracted from the .sqlite file, we don't know if the entries refer to domains visited by the user or domains simply "trusted/untrusted" by the user when visiting something else.


After several trial and error attempts, I've noticed it's possible to distinguish between "visited sites" and "sites not visited" by using HTTP requests and observing the responses. It's possible that the user has visited a domain if the domain:
  • returns a HTTP response
  • doesn't redirect to another domain
  • doesn't have a very small Content-Length size
Moreover, if a domain loads scripts residing on another domain, it's highly probable that the former is the domain that was directly visited by the user.

If all or some of the above conditions are not met, then it's possible that the user has set a domain to "trusted" or "untrusted" when visiting any of the visited domains.

The script I've developed can be downloaded here: Firefox_NoScript.py.
If the script is used with a "-r" option, it will send a HTTP request to each site found in the file and try to separate the results. The more entries there are in the .sqlite file, the more accurate the script output is.


C:\>Firefox_NoScript.py --default-path -r

Firefox_NoScript v.20190127
Script to extract the permissions that have been manually added to NoScript add-on

Analyzing file: C:/Users/xxxx/AppData/Roaming/Mozilla/Firefox/Profiles/
xxxx.default/storage-sync.sqlite ...

Non-default permissions found: (6)

Sending HTTP requests to 6 domains found in the file...

   Based on the HTTP responses received, it's possible that:
     ==> the user directly visited 1 domain(s):
      - cnn.com

     ==> the trust level for 5 domain(s) was set by the user
     when visiting other domains:
      - chartbeat.com
      - cookielaw.org
      - optimizely.com
      - postrelease.com
      - sharethrough.com

Output saved to: 20190127_141136_NoScript.csv

The Tor Browser (8.0.4) comes with NoScript (10.2.1) pre-installed. The .sqlite file is located at:
  • C:\Users\<username>\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default\storage-sync.sqlite
Differently from Firefox, and based on my tests, the storage-sync.sqlite file will retain NoScript last session's settings when the Tor Browser is closed. The settings are lost when the Tor Browser is reopened.


Thursday, December 6, 2018

What was my IP? Ask DoSvc on Windows 10

Introduction
   I recently watched the recording of the interesting talk Windows Forensics: Event Trace Logs that Nicole Ibrahim gave at SANS DFIR Summit 2018. I then used the tool ETLParser to dump the contents of all the ETL files stored on my own workstation. I glanced through the giant CSV output file looking for anything of interest and accidentally noticed a string "ExternalIpAddress". As you can see, the string is followed by an IP address.


That IP address is my current public IP address. Why is it there?


The string "ExternalIpAddress" is located next to other interesting strings like "GEO: response", "CountryCode" and a precious timestamp. If this is a geolocation response, what triggered it? Since "ExternalIpAddress"appears several times in the log files, how many geolocation requests have been made so far and why?


DoSvc - Delivery Optimization
   All the hits were found within some logs whose names begin with "dosvc". I searched on Google and found out that "dosvc" stands for Delivery Optimization which is the update delivery service for Windows 10 clients. In the online documentation Optimize Windows 10 update delivery, Microsoft explains what this service does:

Delivery Optimization is a new peer-to-peer distribution method in Windows 10. Windows 10 clients can source content from other devices on their local network that have already downloaded the updates or from peers over the internet.

Depending on the version of Windows 10, the various Event Trace Log (ETL) files created by the Delivery Optimization service (DoSvc) are stored here:

OS Version Default path Filename
Win10 (1507)  C:\Windows\Logs\dosvc dosvc.\d*.\d.etl
(e.g. dosvc.1377765.1.etl)
Win10 (1709/1803)  C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Microsoft\Windows\DeliveryOptimization\Logs dosvc.yyyyMMdd_HHmmss_\d*.etl
(e.g. dosvc.20181111_180339_399.etl)

On my computer running Win10 (1803), which is always On and connected to the internet 24/7, the default DoSvc log path contains each day about 140 log files. Based on what I've observed, Win10 (1803) daily removes the ETL files older than 57/58 days. In such a scenario, there may be the chance of extracting several public/external IP addresses from the logs.

Even though the log files lead in the direction of "Delivery Optimization", I think something else might be responsible for the geolocation calls. On my computer, the "Delivery Optimization" service is off. Even the "Location" service is turned off.

(Delivery Optimization)
(Location)


How to parse the logs
   After some trial and error trying to figure out what was the best way to extract the data I needed from the CSV output file created by ETLParser, I found out that Win10 has a built-in Powershell cmdlet named "Get-DeliveryOptimizationLog":
This cmdlet retrieves decoded logs for Delivery Optimization. If no parameter is given, the cmdlet parses the default DoSvc path. The parameter "-Path" is required to parse other locations:

Get-DeliveryOptimizationLog -Path C:\CustomPath\*

Here I used the cmdlet to search for the keyword "ExternalIpAddress".

PS> Get-DeliveryOptimizationLog | Where-Object Message -Like "*ExternalIpAddress*"

The output was:
(27/Nov/2018 11:05:47)

I also spotted the IP that I was assigned to when using CyberGhost VPN.

(24/Nov/2018 23:26:39)

What triggered the geolocation requests? Using the two examples shown above, I searched for "ProcessId" 13104 and 36192 and noticed that some events contain the message: "Create job name = WU Client Download".

TimeCreated : 27/11/2018 11:05:47
ProcessId   : 13104
ThreadId    : 9952
Level       : 4
LevelName   : Info
Message     : Create job name = WU Client Download, jobId = 4d66d186-68e2-4bfc-8d74-f40de415fc20, type = 0. hr = 0
Function    : CDeliveryOptimizationManager::CreateJob
LineNumber  : 495

TimeCreated : 24/11/2018 23:26:43
ProcessId   : 36192
ThreadId    : 33560
Level       : 4
LevelName   : Info
Message     : Create job name = WU Client Download, jobId = bc698001-4916-4c93-b513-cdcfe325ae9d, type = 0. hr = 0
Function    : CDeliveryOptimizationManager::CreateJob
LineNumber  : 495

What was downloaded and probably installed by the "Windows Update" (WU) client?

PS> Get-WuaHistory | Format-Table

Get-WuaHistory is a third party cmdlet.

This seems to be the answer:

Result    Date                Title                                                                                                       
------    ----                -----                                                                                                       
Succeeded 27/11/2018 11:04:59 Definition Update for Windows Defender Antivirus - KB2267602 (Definition 1.281.899.0)                       
Succeeded 27/11/2018 11:04:59 Definition Update for Windows Defender Antivirus - KB2267602 (Definition 1.281.899.0)                       

Succeeded 24/11/2018 23:25:54 Definition Update for Windows Defender Antivirus - KB2267602 (Definition 1.281.756.0)                       
Succeeded 24/11/2018 23:25:54 Definition Update for Windows Defender Antivirus - KB2267602 (Definition 1.281.756.0) 

I see from my Windows Update history that Windows Defender Antivirus is updated on a daily basis. Based on the log files, it seems that "WU Client" makes a geolocation call before downloading any available update. That could explain why I have at least one geolocation response per day in the logs. Additionally, the creation time (TimeCreated) of each "GEO: response" event message always matches or is very close to the installation date of each antivirus definition update. 

A hive file named "dosvcState.dat" is also involved in the process, but I haven't had the time yet to check what it contains. The hive can be found here:

C:\WINDOWS\ServiceProfiles\NetworkService\AppData\Local\Microsoft\Windows\DeliveryOptimization\State\dosvcState.dat 


Scripting
   I wrote a Powershell script that adapts to my needs the output provided by the mentioned cmdlet. The script adds to the output the name of the log files from which the information was extracted and shows the contents of the "Message" object in a different way. The script will generate two output files in both CSV and JSON format:
  • <timestamp>_dosvc_ExtIpAddress: contains the extraction of each IP found in the ETL files;
  • <timestamp>_dosvc_ip2location: contains additional details about each unique IP found like the Internet service provider, latitude and longitude. The script uses an external API.
Filenames are prepended with the timestamp of when the script was executed. To use the script, just provide the path containing the ETL files to parse:

PS> .\Get-DoSvcExternalIP.ps1 C:\LogPath

From the logs of the computer I mentioned above (Win10 - 1803), I managed to extract 124 IP addresses whose dates range from October 10th 2018 to yesterday.

I also used the script against the DoSvc ETL files that I extracted from a laptop (Win10 - 1709) that I last used in May in Las Vegas during the Magnet User Summit 2018. I connected to the Wi-Fi network in the hotel a couple of times for a short moment, but that was long enough for Win10.
This is an example of what I could extract from the logs:

PS> (Get-Content 20181205_143620_dosvc_ExtIpAddress.json | ConvertFrom-Json) | Where-Object ExternalIpAddress -eq "xx.xxx.xx.x6"

LogName                  : C:\TEMP\ETL_Laptop2\dosvc.20180522_170803_773.etl
TimeCreated              : 22/05/2018 17:09:04
ExternalIpAddress        : xx.xxx.xx.x6
CountryCode              : US
ProcessId                : 7840
ThreadId                 : 776
Level                    : 4
LevelName                : Info
KeyValue_EndpointFullUri : https://kv801-prod.do.dsp.mp.microsoft.com/all
Version                  : <omissis>
Function                 : CGeoInfoProvider::RefreshConfigs
LineNumber               : 58

These are the additional details provided by the script by using an external API:

C:\> type 20181205_143620_dosvc_ip2location.json | jq

  {
    "as": "ASxxxxx Cox Communications Inc.",
    "city": "Las Vegas",
    "country": "United States",
    "countryCode": "US",
    "isp": "Cox Communications Inc",
    "lat": xx.x892,
    "lon": -xxx.x63,
    "org": "HOSPITALITY NETWORK, LLC",
    "query": "xx.xxx.xx.x6",
    "region": "NV",
    "regionName": "Nevada",
    "status": "success",
    "timezone": "America/Los_Angeles",
    "zip": "89106"
  },

I hope you find this long blog post useful!

You can download the script from my GitHub repository here.


[UPDATE March 31, 2019]:
the peer-reviewed version of this article can be read at DFIR Review.

[UPDATE April 5, 2019]:
I improved the script and created a "Get-DoSvcExternalIP" module for KAPE.

Sunday, June 10, 2018

UsrClass.dat stores more history than you think

This is a quick post about two new plugins I wrote for RegRipper that will pull the following artifacts from a Windows 10 UsrClass.dat hive:

  • Microsoft Edge web history (plugin msedge_win10.pl)
  • Microsoft Photos recent file history (plugin photos_win10.pl)

The plugins will parse the following keys:

Microsoft Edge
  • Local Settings \ Software \ Microsoft \ Windows \ CurrentVersion \ AppContainer \ Storage \ microsoft.microsoftedge_8wekyb3d8bbwe \ MicrosoftEdge \ TypedURLs
  • Local Settings \ Software \ Microsoft \ Windows \ CurrentVersion \ AppContainer \ Storage \ microsoft.microsoftedge_8wekyb3d8bbwe \ MicrosoftEdge \ TypedURLsTime
  • Local Settings \ Software \ Microsoft \ Windows \ CurrentVersion \ AppContainer \ Storage \ microsoft.microsoftedge_8wekyb3d8bbwe \ MicrosoftEdge \ TypedURLsVisitCount

Microsoft Photos
  • Local Settings \ Software \ Microsoft \ Windows \ CurrentVersion \ AppModel \ SystemAppData \ Microsoft.Windows.Photos_8wekyb3d8bbwe

Here are some output examples:

msedge_win10 v.20180610
(USRCLASS.DAT) Get values from the user's Microsoft Edge Windows App key

|-- \Local Settings\Software\Microsoft\Windows\CurrentVersion\AppContainer\Storage\microsoft.microsoftedge_8wekyb3d8bbwe
|----- \MicrosoftEdge\TypedURLs
|----- \MicrosoftEdge\TypedURLsTime
|----- \MicrosoftEdge\TypedURLsVisitCount

url1 (TypedURLs)           -> https://www.google.it/
url1 (TypedURLsTime)       -> Tue Jan  2 17:19:53 2018 (UTC)
url1 (TypedURLsVisitCount) -> 4

photos_win10 v.20180610
(USRCLASS.DAT) Get values from the user's Microsoft Photos Windows App key

Local Settings\Software\Microsoft\Windows\CurrentVersion\AppModel\SystemAppData\Microsoft.Windows.Photos_8wekyb3d8bbwe\Schemas
  PackageFullName => Microsoft.Windows.Photos_2017.37071.16410.0_x64__8wekyb3d8bbwe

Local Settings\Software\Microsoft\Windows\CurrentVersion\AppModel\SystemAppData\Microsoft.Windows.Photos_8wekyb3d8bbwe\PersistedStorageItemTable\ManagedByApp

 {2091DC0D-FB11-4834-8ECB-E9F628131FA8}
   KeyLastWrite   : Sat Jun  9 14:50:21 2018 (UTC)
   LastUpdatedTime: Sat Jun  9 14:06:02 2018 (UTC)
   Metadata       : StartFileC:\Users\username\Desktop\3rd.jpg

## Microsoft Photos (Windows App): Recent Files ## (Tab-separated values)

StartFileC:\Users\username\Desktop\3rd.jpg       KeyLastWrite: Sat Jun  9 14:50:21 2018 (UTC)

The tests were done with registry hives exported from computers running Windows 10 version 1511 and 1709.

The scripts are available for download here on my GitHub page.

Let me know if you find any other interesting app storing history activity within this registry hive. There's more than just shellbags inside UsrClass.dat!


References

Saturday, April 1, 2017

RegRipper plugin to parse Foxit Reader

Foxit Reader is a popular free PDF Reader. Like any other program it keeps a file history. This is the Recent Files list shown to the user when the program is launched:


Foxit Reader stores the MRUs under the File MRU and Place MRU subkey in the NTUSER.DAT hive. They are named Item1 – Item50 and hold up to 50 of the last PDF file/path opened. Item 1 contains the most recent value while Item 50 is the oldest last. History\LastOpen contains additional details that I'll mention later.


As a test, I opened 50 different files named with numbers: I first opened a file named 01.pdf, then 02.pdf and so on up to 50.pdf.


Under History\LastOpen, Foxit Reader stores for each file some important information like the page number of the last page read, the zoom level used and the view mode. As you can notice from the picture below, the page number counter starts at "0". That means that page number "1" in a PDF file is stored as "0" in the registry.


I wrote a plugin for RegRipper to parse all these values by adapting a couple of existing plugins (adoberdr.pl by H. Carvey and iexplore.pl by E. Rye). The code I wrote is far from perfect since I've never programmed in Perl...but it works ;) 

My foxitrdr.pl plugin can be downloaded from here.

-----------------------------------------------
[UPDATE 11/April/2017]: My plugin was added to the official RegRipper repository. Thanks Harlan!

Friday, March 31, 2017

Customizing the filter type in X-Ways Forensics

The Filter:Type in X-Ways Forensics is one of my favorite filters. After many uses, I started thinking on how to make it more suitable for my needs.

This is my tweaked version:


This is a quick summary of the changes:
  • the categories are sorted alphabetically;
  • some categories were renamed;
  • there are now new categories like Network/Packets and Memory;
  • some extensions were moved to other categories;
  • some new extensions/filenames were added to the list.

If you want to give it a try, replace the two files "File Type Categories.txt" and "File Type Categories User.txt" in your installation folder with the ones you can download from my repository xways-forensics .

References

-----------------------------------------------
[UPDATE 03/April/2017]: I added the category Malware, Ransomware which is based on the Ransomware Overview document.
-----------------------------------------------
[UPDATE 11/July/2017]: The custom filter types was added to the Bookmarks menu in the latest version of XWFIM X-Ways Forensics Installation Manager (v1.7.0.0). Thanks Eric!

Saturday, August 15, 2015

Geotag2kml: python script to create a KML file from geotagged pictures

Sometimes it's not the photo itself that matters, but where the photo was taken.

I needed a tool to parse thousands of geotagged pictures and show them on Google Earth. I wrote a Python script based in part on what was posted years ago in the ExifTool Forum.

My script was written to:
  • parse recursively geotagged pictures
  • create a KML file to show geotagged pictures on Google Earth
  • group and sort GPS data by date
  • show visually for each date where the first geotagged picture was taken. Each first GPS point is indicated by the icon of a small man
  • connect the GPS points of each date with a colored line
  • get the preview of a picture when clicking on a placemark
  • list make and model information of each digital camera used to take and geotag the analyzed photos
  • speed up my analysis ;)

Prerequisites
  • Python v2.7
  • Exiftool (rename the executable to "exiftool.exe" and put it in the same folder of the script)
  • Google Earth

Usage

Run the script and type the absolute path of the directory containing your pictures. The script will create and save in this path a file named "GoogleEarth.kml".

Download

geotag2kml v0.1

Here are a couple of screenshots.





Thursday, August 13, 2015

USB Write-Blocking with the registry: Beware of UASP on Windows 8/8.1 - Workaround

I read once more the description of USB Attached SCSI on Wikipedia and I noticed these two lines:

Microsoft added native support for UAS to Windows 8. Drives supporting UAS load Uaspstor.sys instead of the older Usbstor.sys. Windows 8 supports UAS by default over USB 2.0 as well.

That explains why my UASP device works in "UASP" mode even if I plug it in on a USB 2.0 port.

I then thought: what if I replace uaspstor.sys with usbstor.sys?

And you know what? It worked!



These are the steps for the workaround:

  • boot into safe mode: within Windows, hold the SHIFT key and click Restart
  • click on the Troubleshoot button
  • select  Advanced Options
  • choose Command Prompt
  • login into your admin account
  • from command prompt, type C: and press Enter
  • type cd windows\system32\drivers and press Enter
  • type ren uaspstor.sys uaspstor.sys.old and press Enter to rename the file uaspstor.sys into uaspstor.sys.old (or whatever you like)
  • type copy usbstor.sys uaspstor.sys and press Enter to have an additional copy of usbstor.sys renamed into uaspstor.sys
  • close your command prompt by clicking on the "X" in the upper right corner
  • click on the Continue button to exit and reboot into Windows 8/8.1

Set "WriteProtect" to "1" in the registry, plug in your UASP device and finally enjoy it in read-only mode!

Since I applied the workaround, I haven't had any BSOD or software issue. So far it appears to be a stable workaround.

Your feedback is appreciated, thanks.

Sunday, August 9, 2015

USB Write-Blocking with the registry: Beware of UASP on Windows 8/8.1

Introduction

On Microsoft Windows operating systems it's possible to use the Windows registry to disable write access on USB ports.

Figure 1: HKLM\SYSTEM\CurrentControlSet\Control\StorageDevicePolicies

That has been for a long time a convenient and safe way to make USB ports read only. Until UASP came out.

"USB Attached SCSI Protocol" (UASP) is designed to improve USB 3.0 transfer speeds. Microsoft Windows 8 has native support for UASP as written on the page "Windows 8: What's new for USB":
Windows 8 includes a new USB storage driver that implements the USB Attached SCSI Protocol (UASP). The new driver uses static streams for bulk endpoints, as per the official USB 3.0 specification.
Analysis

For this test I decided to use:
  • HP EliteBook 8470w Mobile Workstation
  • OS Windows 8.1 Pro x64
  • Transcend JetFlash USB 3.0 64GB Flash Drive (not UASP)
  • StarTech adapter cable USB 3.0 to 2.5" SATA  w/UASP  (mod. USB3S2SAT3CB) with HGST 1 TB internal 2,5" SATA drive
After turning ON the USB write protection with the Windows registry, I plugged in the two USB 3.0 devices directly into my laptop.

The tool USB Device Viewer shows that the flash drive has been recognized as a "USB Mass Storage device", while the other one as a "USB Attached SCSI mass storage device".

Figure 2: output of USB Device Viewer

In Windows PowerShell, I used the cmdlet GET-WMIOBJECT to list my drives:

PS > GET-WMIOBJECT win32_diskdrive

The thumb drive is shown as a "USB device", on the other hand the SATA drive (externally connected via USB) is shown as a "SCSI disk device". My notebook has USB 3.0 and USB 2.0 ports, but it doesn't make a difference where I plug in the drive. The adapter makes the SATA drive appear as a SCSI device.
Figure 3: Powershell "GET-WMIOBJECT win32_diskdrive" on Win 8.1 Pro

I then checked the read-only state of the two devices by using Diskpart.

The thumb drive (Disk 3 - \\.\PhysicalDrive3) is in read-only mode as expected.

Figure 4: the thumb drive is in read-only mode

The external SATA drive (Disk 2 - \\.\.PhysicalDrive2) is NOT in read only mode.

Figure 5: the external SATA drive is still in WRITE mode
I successfully created a new folder named "BrandNewFolder".

Figure 6: folder creation

 And I created a new txt file named "BrandNewFile.txt" inside this folder.

Figure 7: file creation

I then unplugged the drive and I plugged it in on a second computer (Intel NUC DN2820FYKH with Windows 8.1). The newly created folder was still there. Unfortunately that means I modified my "evidence" drive.

Figure 8: the folder was really written to the drive

I repeated the same test on the second computer and I had the same results.

I made a last test: I went back to my laptop and installed Windows 7 Pro (on a different internal drive). Windows 7 has no native support for UASP. I repeated all the steps written above and this time the 1 TB drive with the mentioned adapter was recognized as a simple USB device in read only mode.

Figure 9: external drive in read-only mode on Windows 7 Pro


Conclusion

The registry key doesn't work on Win8/8.1 with UASP devices. At the moment I haven't found a way to disable UASP. I googled a bit and have found out there are already around some thumb drives which use UASP.

For the time being, stay safe on Windows 7 or choose a hardware write-blocker.