Tag Archives: databending

How to datamosh videos with automation

Datamoshing videos can be a time-consuming process, automation can help. For Windows users AutoHotkey is free, open-source macro-creation and automation software that can handle some of the repetitive tasks involved in datamoshing.

The following script for AutoHotkey automates I-frame removal in Avidemux, normally a manual process described in this tutorial. The video above was datamoshed using this automation script.

Load AutoHotkey with the script below and then when it comes time to remove I-frames in Avidemux simply focus the slider below the video and press Control+F to trigger the AutoHotkey script. The script will send the appropriate key strokes to remove the next 10 I-frames while you pop out for a break.

; Remove next 10 I-frames
^f::
Loop 10 {
	Send,{Up}
	Sleep, 500
	Send,[
	Sleep, 500
	Send,{Right}
	Sleep, 500
	Send,]
	Sleep, 500
	Send,{Delete}
	Sleep, 1000
}

These types of scripts could also be used to automate key strokes while hex editing images, consider a script which would move a certain number of characters across and then insert a character — that could glitch out an image quite nicely. Similarly one could experiment with automating photo editing processes by scripting with a program like AutoHotkey.

Some of these types of automation could be accomplished through the usage of a programming framework, or scripting language, but automating at the user interface level can remove a lot of overhead and restrictions.

Windows

How to glitch images using RGB channel shifting

Channel shifting is the process of taking the red, green or blue values of pixels in an image and applying those values to pixels in different positions on the image. In this tutorial we are going to accomplish this effect using the Processing language.

If you don’t have the time or inclination to glitch images using scripts you can use dedicated apps such as Glitch for iOS.

glitch4ios

To get started download and install the latest version of Processing, version 3.1.1 at the time of writing this. I’ve written a channel shifting script you can download here, you’ll have to unzip it once it’s downloaded. Once you’ve installed and opened Processing you can load the script by accessing the menu.

File > Open

And navigating to the ChannelShiftGlitch.pde script file. In the script, which are referred to as sketches in Processing, you’ll need to change the following lines to point the script at the image you want to glitch. It’s easiest to place the image in the same directory as the script file.

// image path is relative to sketch directory
String imgFileName = "MyImage";
String fileType = "jpg";

I’ve set up some settings variables to make the script easier to use, you can see these towards the top of the script under the comment of script settings.

// repeat the process this many times
int iterations = 5;

// use result image as new source for iterations
boolean recursiveIterations = false;

// shift the image vertically true/false
boolean shiftVertically = false;

// shift the image horizontally true/false
boolean shiftHorizontally = true;

This script is able to apply the same channel shifting effect multiple times, the number of times is specified by the iterations variable, currently set to 5. This variable drives a for loop around the channel shifting code as seen below.

// repeat the process according 
// to the iterations variable
    for(int i = 0;i < iterations;i++)
    {
      // generate random numbers 
      // for which channels to swap
      int sourceChannel = int(random(3));
      int targetChannel = int(random(3));

You can also see in this code where the script generates a random number which will determine which of the three channels, red, green or blue are used as a source, and which of the three channels are used as a target. Next the script sets up the shifting positions, how far vertically and how far horizontally the channel should be shifted. These are either 0 if shifting is set to false for that plane (determined by the shiftHorizontally and shiftVertically settings), or a random number between the 0 and the height or width of the image.

// start with no horizontalShift 
int horizontalShift = 0; 

// if shiftHorizontally is true generate a 
// random number to shift horizontally by
if(shiftHorizontally)
  horizontalShift = int(random(targetImg.width));
      
// start with no verticalShift 
int verticalShift = 0;
      
// if shiftVertically is true generate a 
// random number to shift vertically by
if(shiftVertically)
  verticalShift = int(random(targetImg.height));

Next the script calls the main copyChannel method. This method accepts pixel arrays of the source and target images and will copy one channel to another from one part of the image to another and wrap around both horizontally and vertically if it runs out of space.

// shift the channel
copyChannel(
  sourceImg.pixels, 
  targetImg.pixels, 
  verticalShift, 
  horizontalShift, 
  sourceChannel, 
  targetChannel
  );

The method starts by starting a counter to loop through the rows of pixels in the image, top-to-bottom. This counter is added to the sourceYOffset variable to apply the vertical shift. If the vertical shift plus the counter is higher than the image height we subtract the image height to wrap the shift around to the top of the image.

// starting at the sourceY and pointerY
// loop through the rows
for(int y = 0; y < targetImg.height; y++) { 
  // add y counter to sourceY 
  int sourceYOffset = sourceY + y; 

  // wrap around the top of the 
  // image if we've hit the bottom 
  if(sourceYOffset >= targetImg.height)
    sourceYOffset -= targetImg.height;

Within the row loop the script starts another counter to loop through the columns in that row, left-to-right. It also adds that counter to the sourceXOffset to apply the horizontal shift. If the horizontal shift plus the counter is wider than the image width we subtract the image width to wrap the shift around to the left of the image.

// starting at the sourceX and pointerX 
// loop through the pixels in this row
for(int x = 0; x < targetImg.width; x++) 
{ 
  // add x counter to sourceX 
  int sourceXOffset = sourceX + x; 

  // wrap around the left side of the 
  // image if we've hit the right side 
  if(sourceXOffset >= targetImg.width)
    sourceXOffset -= targetImg.width;

Processing stores image pixels in an array as illustrated in the image below.

pixelarray

In order to access a pixel at specific x/y coordinates in the image we use the formula below.

y * width + x

Next the script isolates the RGB (red, green, blue) values for both the source and target pixels by using the formula above to access the pixel and then some Processing methods to extract the separate RGB channel values.

// get the color of the source pixel
color sourcePixel = 
  sourcePixels[sourceYOffset * targetImg.width + sourceXOffset];
            
// get the RGB values of the source pixel
float sourceRed = red(sourcePixel);
float sourceGreen = green(sourcePixel);
float sourceBlue = blue(sourcePixel);
   
// get the color of the target pixel
color targetPixel = targetPixels[y * targetImg.width + x]; 

// get the RGB of the target pixel
// two of the RGB channel values are required 
// to create the new target color
// the new target color is two of the target
// RGB channel values and one RGB channel value 
// from the source
float targetRed = red(targetPixel);
float targetGreen = green(targetPixel);
float targetBlue = blue(targetPixel);

Now that the script has the RGB of the source pixel and RGB of the target pixel we can proceed to shift one of the channels. We use a switch statement for this, deciding which source channel to use based on the sourceChannel variable which has a holds a random number we generated earlier, either 0, 1 or 2.

// create a variable to hold 
// the new source RGB channel value
float sourceChannelValue = 0;
            
// assigned the source channel value 
// based on sourceChannel random number passed in
switch(sourceChannel)
{
  case 0:
    // use red channel from source
    sourceChannelValue = sourceRed;
    break;
  case 1:
    // use green channel from source
    sourceChannelValue = sourceGreen;
    break;
  case 2:
    // use blue channel from source
    sourceChannelValue = sourceBlue;
    break;
}

After selecting a source channel we apply that channel value to either the red, green or blue channel of the target pixel, again using a switch statement, this time based on the targetChannel variable.

// assigned the source channel value to a 
// target channel based on targetChannel 
// random number passed in
switch(targetChannel)
{
  case 0:
    // assign value to target red channel
    targetPixels[y * targetImg.width + x] = 
      color(sourceChannelValue, 
        targetGreen, 
        targetBlue);
    break;
 case 1:
    // assign value to target green channel
    targetPixels[y * targetImg.width + x] = 
      color(targetRed, 
        sourceChannelValue, 
        targetBlue);
    break;
 case 2:
    // assign value to target blue channel
    targetPixels[y * targetImg.width + x] = 
      color(targetRed, 
        targetGreen, 
        sourceChannelValue);
    break;
}

That’s it for the copyChannel method. The channel has been shifted in the target image at this point. Back in the main draw method of the script there is an if statement that determines whether or not the next iteration (if the iterations variable is to to greater than 1) will use the original image as a source, or use the new shifted image as a source.

// use the target as the new source 
// for the next iteration
if(recursiveIterations)
  sourceImg.pixels = targetImg.pixels;

Using the original image as a source for more than 3 iterations is rather pointless because there are only three channels in the original image to shift to around, always resulting in three shifted ghost images. So if you set iterations higher than 3 you should probably set recursiveIterations to true.

Vertical-Channel-Shift

Setting the recursiveIterations variable to true at the beginning of the script will use each new shifted image as a source for the next iteration and will result in much more dynamic results when iterations is set higher than 3, say 25 or 50.

Recursive-Channel-Shift

Personally I prefer restricting the shifting to either horizontal or vertical alone, but the script allows for the combination  by changing the shiftVertically and shiftHorizontally settings. You can find more Processing tutorials here, and remember, if you’re going to corrupt, corrupt absolutely. #corruptabsolutely

iOS (iPhone, iPad)

Windows

OSX

Processing

How to datamosh videos with data corruption


Glitching videos with data corruption can be a tricky process. This is owing to the fact that video formats are substantially more complex than image formats. Since video formats contain audio and timing information in addition to visual information corrupting the wrong section of a video file can quickly render the file unplayable rather than delightfully distorted.

This tutorial will focus on glitching the popular MP4 and MOV formats containing video compressed with the H.264 standard. If the video you want to destroy is not in MP4 or MOV format already an easy way to convert it is to upload it to YouTube, let them convert it, and then download the result. If a video doesn’t set off any copyright claims you can download it from the YouTube Video Manager in MP4 format.

Get started by making a copy of your MOV or MP4 and open the copy in a hex editor — never edit the original file. If you don’t have a hex editor installed there are some freeware options listed at the bottom of this post. Hex editors allow us to view and edit the bytes of a file using hexadecimal. Editing the file using hex rather than text allows greater flexibility since we’re no longer restricted to text characters (which are each represented by two hex digits).

We know that we’re looking at an MP4 or MOV file when we see the text ftypqt starting on the fifth byte of the file as illustrated in the example below.

hex-ftypqt

The MP4 and MOV (Quicktime) formats utilize a similar structure, the file is broken down into atoms or blocks of data. The atom which contains the raw frame and audio data can be identified by it’s atom type string, which in this case is mdat (short for media data).

mov mp4 hex mdat

The data contained within the mdat atom is comprised of chunks, which are comprised of nal units, which are comprised of slices. For the purposes of this tutorial we won’t delve that deep. Finding the mdat atom can be done by searching the file for the string “mdat” as seen in the above image. Notice that the contents of the mdat atom seem quite random when viewed in a hex editor, in contrast the contents of the other atoms in MP4 and MOV files are very structured. With this information we can easily find the end of the mdat atom by scrolling through it (or up from the bottom of the file) to see where the data starts to look random as illustrated in the image below.

hex-moov

Here we can see that the mdat atom is followed by the moov atom (and an mvhd atom after that), this is not always the case as the order of the atoms can be different. What’s important to note though is that the file is visibly more structured after the mdat atom, this is how we can identify where the mdat atom ends. The mdat atom, in all cases, will either continue to the end of the file or it will be followed by another atom identified by a 4-character string such as the moov atom does in the above example. Using this method we can identify both the start and the end of the mdat atom, and in turn where we can corrupt only the mdat atom’s contents and have a fair chance of the video still being playable.

Once we’ve identified the boundaries of mdat atom we can begin to copy and paste, replace or edit portions of the raw hex data (or the text ASCII data, either will work) of the video and check the result along the way by attempting to play the video. Making backup copies after every successful change will avoid heartaches when, not if, a misstep renders the video unplayable.

Some notes on successful corrupting:

  • It doesn’t take much corruption to add bizarre distortion to a video, even corrupting as little as 10% of a file, a couple of bytes here and there, has the potential to send playback into a wild frenzy.
  • While not required, most data in the mdat will be in sequence so if we want to corrupt a specific portion of the video we can estimate the offset of the data for that portion is in the mdat based on its time.
  • The mdat atom will also contain raw audio data so if the audio becomes distorted during playback we know we’ve gone too far, or started too early, in the mdat atom.
  • Copying and pasting hex within nal units is probably the best way to corrupt H.264 video data as you’ll be shuffling valid data around rather than adding outright gibberish.
  • As mentioned previously the mdat is comprised of chunks, as chunks and the nal units contained within them have structure it’s best to corrupt small portions of data in various spots rather than large swaths. Corrupting large regions of data will inevitably cross over structure boundaries and destroy important information regarding the type of nal unit or slice.
  • To stay inside nal units look for hex 67, 68 or 00 00 01 as these sequences can denote the beginning of a new nal unit.
  • This definitely falls under the bull in a china shop category of datamoshing video so don’t be discouraged if it takes a couple of restarts to get a playable result.

The video included in this post was glitched using this technique, however the audio was slowed down using traditional video editing.

As with any glitch-by-corruption technique, it requires a soft touch — too little has no effect, too much can destroy the file, but just enough results in glorious, glorious corruption. #corruptabsolutely

Windows

OSX

How to glitch images using audio editing software

Images can be pleasantly destroyed in a great number of ways, some of the best results come from applying transformation algorithms to the raw image data. Applying filter algorithms to images is something one would normally use software like Adobe Photoshop for, however, using audio processing software instead can yield much more interesting, and unexpected, results.

If you don’t have the time or inclination to manually glitch images with audio processing software you can use dedicated apps such as Glitch for iOS.

glitch4ios

Firstly we’ll need some audio processing software, many will work, Audacity is free, supports many platforms and works quite well for glitching images. Secondly we’ll need a databending-friendly image, the BMP format works well for this type of bending. Once we have an image and Audacity installed, open Audacity and import the image by selecting:

File > Import > Raw Data

audacity-importraw

Audacity will then ask for some information about the file we are importing, we’re going to lie, for Encoding select either U-Law or A-Law. We will have to export with the same encoding setting so remember which was selected. Defaults will work fine for the rest of the import options.

audacity-import

The image will now be open as an audio file, I don’t suggest pressing play. Now we can select any portion of the file or its entirety by clicking and dragging on the waveform (the chart-like display). In some cases it’s better avoid selecting the beginning (first 5-10 seconds of the waveform) of the file as this contains the file header, a section of the file which contains information needed to display the image, if the image won’t display after exporting consider leaving the header intact.

Once we have a selection, we can apply any of the filters under the Effects menu. I have found the Invert, Reverb, Reverse, Wahwah, Compressor and Echo work quite well, but here is where you can experiment. You’ll see the waveform change as each filter effect is applied. The hero image of this post was created using the Compressor and Echo filters applied to the entire file.

Once we’ve applied one or more filter effects we can export the data back to an image by selecting:

File > Export Audio

Change the filename back to the proper image extension, in this case BMP. The Save as type should be set to Other uncompressed files, the Header should be set to RAW (header-less) and the Encoding should be set to either U-Law or A-Law depending on which was chosen during the import process.

audacity-export

Audacity may complain that not all is well with the filename, and perhaps prompt for metadata but these prompts can be accepted and ignored.

All that’s left is to check the result — in an image viewer, not a music player. #corruptabsolutely

Audacity
Audacity Forums

How to glitch JPG images with data corruption

Glitching JPG (or JPEG) images by corrupting their data is a relatively straight-forward affair. Simply open up the file in a hex editor and wreck up the place. Corrupted JPG images can be identified by the telltale offset horizontal bands of changing hues and small square sequences of artifact patterns at the beginning or end of these bands.

If you don’t have the time or inclination to manually glitch JPG images you can use dedicated apps such as Glitch for iOS.

glitch4ios

If you don’t have a hex editor installed there are some freeware options list at the bottom of this post. Hex editors allow us to view and edit the bytes of a file using hexadecimal. Editing the file using hex rather than text allows greater flexibility since we’re no longer restricted to text characters (which are each represented by two hex digits). Most hex editors display both hex and text (also known as ASCII) in the same view but in separate columns. You can see a JPG open in a hex editor in the image below.

hex-exif

Get started by making a copy of a JPG and open the copy in a hex editor — never edit the original file. The first bytes of a JPG file contain what’s know as the file header. The header contains information that is required for the image to be displayed at all and should be left intact (though feel free to experiment). We need to locate the meat of the file, the raw image data, we can usually tell the raw data apart from the header and other important structural data by it’s garbled nature.

hex-jpg

We can see where some data ripe for glitching is in the above example where the file changes from structured, to seemingly random data. Not to say that there’s no structure, it’s just harder to discern in compressed image data. Once we’ve identified the raw image data we can copy/paste chunks, search/replace sequences or just manually corrupt the data by changing the text (on the right side in the example) or the hex (on the left side in the example) of the raw data.

Now we can begin the process of making changes and checking the result in our favorite image viewer. Making backup copies after every successful change will avoid heartaches when, not if, a misstep renders the image unviewable.

artifacts

Corrupting JPG images often results in interesting patterns due to the corrupt data and the compression algorithms used, as seen enlarged in the example above. Decreasing the quality of the JPG itself, which can be done with image editing software, can sometimes increase the likelihood of generating these artifacts through corruption.

This method can also be used to glitch some other formats as well, most notably BMP files.

As with any glitch-by-corruption technique, too little has no effect, too much can destroy the file, but just enough results in glorious, glorious corruption. #corruptabsolutely

iOS (iPhone, iPad)

Windows Phone

Windows

OSX