Image Space Updates!

Yes I have rebranded it.

 

Jump over to my public repo on github to download it: Scan Space Image Processor

 

First off I want to thank you all for the fantastic response to the image processor. Great to see!

There were a number of standout requests which I got, namely

 

Smarter importing of images

Customised export syntax

Network Processing

Better Masking

Support for a wider range of Colour Charts

Better Automatic Chart Detection

Better UI

Use app without Chart

Image tweaks

 

Some of these are easy, some are hard. I’m not the greatest developer in the world. So for now I am sticking with the easy ones, or features where I have built similar implementations in other software I have made.

Additionally, I welcome people to help develop this. if you have a feature or fix, then please go ahead!

First off lets go through what I have added:

 

Smarter importing of images

Context sensitive importing of images, dataset assignment and grouping

Customised export syntax

Now you can use a simple smart-shooter inspired input to define output path and image naming conventions

Network Processing

There is now a dedicated server for distributing image processing tasks

New Headless client to connect to the server to monitor for new jobs and process

Server status web panel

Settings Menu

Common application settings, such as file format defaults, file naming and export as well as server settings are now available in this menu.

UI and User experience Updates

Better layout of tools and workflow order

Store charts in a folder to be used later

Chartless Processing

White Balance, Exposure, Shadows, Highlights, Sharpening and Denoising added

Better Chart Detection

When using a color checker classic, it will default to a more intensive Machine Learning based detection.

More efficient memory usage

The primary processing code has been reworked to significantly lower the memory requirements allowing for almost 2x the thread count on the same specs

This update has been fairly significant under the hood, with a lot of work being done to make the codebase more readable and documented. Additionally, to service the need of client/server processing I have shifted all code which may be required by the client processors out of the primary application and into their own files.

UI Overhaul

The UI has been overhauled with all main functionality being shifted into a collapsable center column

All of the most used functionality now lives in these panels, give it a try!

 

I have also updated how the image preview works. Thumbnails are now cached and shown, however if you stay on an image for a few seconds it will show a 1mpx version of your image.

 

Chartless Processing

Inside the Colour Chart Tools panel, you will see a new checkbox

Don’t Use Colour Chart

This allows you to skip any chart requirements and exclusively use the new image editing tools

Better Chart Detection

I have now added a drop down menu at the top of the Colour Chart Tools panel.

Select the chart type which you want to detect and run Set Selected as Chart

The detector for Colour Checker Classic will take a wee while longer than the Passport detector, but is far more robust.

Image Editing Tools

Inside the Image Editing tab, you will find some basic sliders. These will adjust the image as per the name of the slider and give you a rather stuttery, but real-time view of the expected output image.

 

There are a couple of minor gotchas with this, such as you need to wait until the image has loaded the full res preview into the preview window before adjusting.

 

Any sliders which get changed will impact your export times, so leaving them blank will produce the fasted image processing speeds.

 

Note that this will have SIGNIFICANT impacts on RAM usage. please test and adjust your thread counts if using this!

 

Note that this is very beta and is incredibly likely to change in the future!

Settings Menu

Almost all of these additions revolve around the added settings menu. This can be found in File > Settings.

Inside this menu there are three tabs, General, Import and Export and Server settings.

The settings are stored to your local machines registry under scanspace, however I may switch to a hard .ini file if there is demand.

General Settings

The general settings tab is where you will find most commonly used variables that you may want to set to a fixed value to be used on startup

Several of the options are currently disabled as I felt they weren’t necessary for this release, however these will come fairly soon.

Display Log

The display log checkbox will simply hide or show the log output. This is great to get more image preview space.

Display Thumbnails with Colour Correction

This option will run the colour correction output on your thumbnails. Application speed will be slowed while this is active, hence why its not done automatically.

Allow Processing without Chart

This option is largely a placeholder, and will skip the colour correction steps and just output images as-is.
This is actually not terrible when paired with the average exposures feature.

(Future plans for lightroom-style sliders for exposure, colour temperature, highlights and shadows are in the planning stage)

Default Export Format

Simply sets the default format to the value in the dropdown, you can also check the ‘16 bit default’ option if this is something you use regularly.

Default Colourspace

Sets the default colourspace for exr export

Use Default Colour Chart

If you are working with images from a scanner with rigid lighting/cameras then you may want to skip the chart detection pipeline. This option will let you set a path to a chart matrix file.

To use this, I recommend you open the image with a chart you want to use, run detect chart and then copy the file it outputs to the log to a folder where you can reference it in the future

 

Smart Import/Export

In Settings > Import and Export you will find the import configuration.

Checking Use Import Rules will let the application import images based on the selected rules.

Look for Images in Subfolders

This option will expand the image search to all subfolders within the selected root folder.

Group Images by Subfolders

When images are stored in subfolders, the subfolder name will become the group name for all images within the folder.

When images are grouped together, they can be assigned their own colour chart.

Note

If there is only one colour chart selected, then that calibration profile will be used for all images in all groups.

Group Images by Filename Prefix

Simply put, this will group images together based on the prefix of the image name.

Say you have a multi camera rig and have cameras outputiing images with the following names:

Cam01_IMG001.cr2, Cam02_IMG001.cr2 etc.

With this flag checked, you can set a separator in the text box, for instance “_”. This will then split the first block of text off and group all images by that string. So now all cameras called Cam01 will be grouped together, same goes for Cam02, Cam03 etc.

Filter Text

This field will force the image loader to Ignore all files/folders containing this text.

This lets you remove any other images which may occur in your folder structure.

Export Rules

The export naming system lets you define exactly how your processed images get named and organized using a simple placeholder system. Think of it like Smart Shooter's naming conventions, but with a bit more flexibility.

The system uses placeholders wrapped in square brackets that get replaced with actual values from your input files. You can combine these with literal text and folder separators to create whatever file structure makes sense for your workflow.

Basic Placeholders

Here's what you've got to work with:

[r] - Root folder name
This pulls the name from your root project folder

[s] - Sub folder name (if it exists)
Great for organizing by lighting setup or camera position

[o] - Original filename (without extension)
The complete original filename minus the file extension

[oc] - Original filename without trailing numbers
Strips off those annoying camera-generated numbers at the end

[c] - Custom name from your input
Whatever you type into the custom name field

[e] - Output file extension
Your chosen export format (.jpg, .png, .tiff, .exr)

[n] - Image number
Sequential numbering for your processed images

[/] - Creates a new folder level
Use this to organize into subdirectories

Padding

You can control how the image numbers get padded:

[n] gives you 1, 2, 3...
[n2] gives you 01, 02, 03...
[n4] gives you 0001, 0002, 0003...
[n6] gives you 000001, 000002, 000003...

The padding is really handy for keeping your files in proper order when you're dealing with large batches, preventing edge cases where you may have 2000 images but only have your padding set to 3 would mean your images, when viewed in a file browser, would be out of order (IE 1000 would show mixed into 1, 1001, 1002 etc)

Filename stripping

The [o], [oc], [r] and [s] placeholder has a couple of extra parameters. These are to allow for filtering text out of your input image name.

Before Separator: [oc-"separator"]

Returns everything before your specified separator.

Say you've got files named "CAM01_IMG001" and you use [oc-"_"] - you'll get "CAM01"

After Separator: [oc+"separator"]

Returns everything after your specified separator.

Same file "CAM01_IMG001" with [oc+"_"] gives you "IMG"

This is particularly useful when you're dealing with multi-camera setups or have specific naming conventions from your scanner.

Examples

Simple Project Naming

Schema: processedImages/[r]_[n4]
Input: ./chineseVase05/IMG_001.NEF
Output: processedImages/chineseVase05_0001.jpg

Organized by Setup

Schema: [r]/[s]/[r][s][n4][e]
Input: ./OldBuilding/crossPolarized/IMG_001.NEF
Output: OldBuilding/crossPolarized/OldBuilding_crossPolarized_0001.jpg

Camera-Based Organization

Schema: [oc-"_"]/[e]/[oc+"_"]_[n4]
Input: ./data/CAM01_IMG001.NEF
Output: CAM01/jpg/IMG_0001.jpg

This one's really useful if you've got multiple cameras all dumping into the same folder and you are wanting to split your images into grouped folders..

Multi-Level Structure

Schema: [r]/[oc-"_"]/[s]/[oc+"_"][c][n4]

Input: ./actor_John/T_Pose/CAM01_IMG001.NEF

Output: skeletonGuy/CAM01/T_Pose/IMG_0001.jpg

Notes:

The extension format, [e], is not required at the end of the schema, this is primarily to allow you to sort your output file formats, such as /output/jpg/image_0002.jpg or /output/tiff/image_0002.tiff

When using the advanced splitting, pick separators that won't appear randomly in your filenames. Underscores are common in camera names, so maybe use something like "_SHOT" or "_IMG" for more reliable splitting.

Avoid the usual problem characters: < > : " | ? *
I have added checks for these characters so you may find you cannot process or import if they exist in your path.

Network Processing

Wowee this is a big one. As this is a fairly big addition there almost certainly will be bugs and other issues that I haven’t caught. So keep that in mind when using this version.

Here is a general overview of the system:

We have three main components. 

1. The Image Processor application itself

2. Server

3. Worker Client

Image Processor 

The Image processor works as normal, however now there is a button at the bottom of the export section for “Send Project to Server”

This will only work if you added the servers IP address in the settings.

Host Server

The Server runs as a standalone launch.

To launch the server, run the following:
python standalone_server.py [--port PORT] [--api-port API_PORT] [--log-level LEVEL]

The arguments are optional.
Default Port: 8888
Default API Port: 8889

Default Log Level: Info

When running, the server will host an interactive web page on http://serverIP:8889

This panel gives you an overview of the currently processing jobs, the queued jobs and the connected clients.

You can adjust the number of processing threads on each connected worker client, as well as force a restart of the worker client in case of hung jobs or other unexpected issues.

Worker Client

The Worker client is launched either via the .bat file, or via commandline:

Python run_headless_processing_client.py [--host HOSTNAME] [--port PORT] [--threads THREADCOUNT]

It has several optional arguments

--host

--port 8888 

--threads

When you first start the client, its best that you give it all of the arguments

Example

python run_headless_processing_client.py --host 192.168.1.100 --port 8888 --threads 4

Once launched, the client will have an interactive terminal GUI, this gives you limited options for changing settings once launched.

The bottom left of the window will list the total images processed, failed images and the how many processing threads are in use

Technical overview