DigitalMigrations

Digital Migrations

Year 1 Class 1

1

Getting Started

Image Glitching

Skill Covered

We are going to use in-browser tools made by the excellent multimedia designer/technologist Georg AKA ‘Snorpey’ that let you manipulate images:

You can get to all of them from here including links to all the javascript code that made them, shared on the popular code sharing platform, github.

Each webpage contains a web application that let’s you drag and drop images from the desktop into the browser window and then distort them and export them. Although there are tonnes of things like this online, Snorpey has open sourced all its simple, contemporary javascript code to refer to on github so should someone want to take this on into their practice it is quite translatable into well established artist friendly coding environments like Processing some interaction software built especially for artists and non-programmers.

Processing itself can be downloaded but also used in a modern Internet Browser as p5js.org.

Image Glitching Exercise

Even if you know very little about digital image manipulation or uninterested in coding you can quickly manipulate and re-manipulate images easily without using more traditional image processing tools like Adobe Photoshop or the open source free alternatives like GIMP in a fraction of the time. Of course these are just 4 image manipulation effects; people use software like Photoshop or Processing to choose exactly what they want. But this is a bit more playful and spontaneous. You could take fragments of your exported images you like and re-work them in physical media; once you start manipulating in software there is no reason to stay on a computer.

Sometimes I think when you invest in time to learn, say Photoshop, Illustrator or Inkscape you end up staying sat at the computer to make it worth it; with an in browser toy you can play then get bored fast: that’s when you can move away from the screen and draw/break/paint/make things. Ill come back to a way of finding and using more diverse tools without coding or learning later here

Gestures are data

After messing about with these webapps, Return to Distorting Grid and notice that in the top right of the browser window is a faint download transform data button. The transformations you make warping the grid etc. are recorded on clicking this, and then exported as a form of data called json.

json

This is just an array or list of data with paired names and values that can be a string of text, numbers or a Boolean (that means either a true or false statement) that humans and machines can read. An example is below. You’d need to use Notepad or TextEdit to read it; it’s just text.

 var myJson = [
 {
    "first": "John",
    "last": "Doe",
    "age": 39,
    "sex": "male",
    "salary": 70000,
    "registered": true
    },
 {
    "first": "Jane",
    "last": "Smith",
    "age": 42,
    "sex": "female",
    "salary": 80000,
    "registered": true
    },
    ];

If you open up the downloaded file with Notepad you’ll see something a bit more abstract that describes the positions of the grid you transformed.

[{
		"points": [
			{
				"x": 0,
				"y": 0
			},
			{
				"x": 40,
				"y": 0
			},
			{
				"x": 0,
				"y": 40
			},
			{
				"x": 40,
				"y": 40
			}
		],
		"transforms": {
			"shape": {
				"scaleX": 1,
				"skewX": 0,
				"scaleY": 0,
				"skewY": 1,
				"translateX": 0,
				"translateY": 0
			},
			"image": {
				"scaleX": 1,
				"skewX": 0,
				"scaleY": 0,
				"skewY": 1,
				"translateX": 0,
				"translateY": 0
			}
		}
	},

There is no mention of your image; that’s because the .json does not really care about it. It just records how you moved the grid in the 2D space using x and y or LEFT/RIGHT, UP/DOWN. The javascript code in the webpage just loads it in an image, maps it to a grid then when you move the grid it stretches the colour and brightness values across the transformation.

The most basic way of representing position is using variables x and y which you’ll remember is a convention for describing horizontal and vertical positions on a graph. We could call them time or chocolate eaten but we only want our position data so it makes sense to use names like x.

We are not going to do anything with it but it shows simply how our interactions manipulating an image with Snorpey’s little apps can be described as data. You could use a tablet and pen and record your hand gesture as easily as a mouse and describe it as JSON. Why? Well machines can then read the data an re-perform your gesture; this old largely archived project Graffiti Markup Language was an attempt to share the gesture data of graffitti artists to make robots perform it or politicians. This project takes a different approach to x and y data, but with lat and long.

2

Another Dimension: 3D object making in the browser

Skill Covered Skill Covered Skill Covered

OpenSCAD is a tool for making 3D and 2D shapes with data and code. Like many things, there’s now a version that runs in a webpage. So we are going to use OpenJSCAD (javascript implementation of OpenSCAD) in the browser, which presents a fairly sophisticated interface for generating 3D shapes

    function main(){
           return cube({size: 10, center: true})
              .translate([-20, -10, 12]);
                  }

DIY dataset

Skill Covered

We’re going to go back to 2 dimensions for a moment and generate more data, not just with our mouse but with our day to day routine. Data does not have to be big. I’m making my own based on times I drink coffee in a 4 hour period in the office.

Im going to plot the times I do that on a graph.

graph

Then Im going to turn it into a list of graph plots

[0, 0], [1, 1], [2, 1], [3, 0], [4, 1], [4, 0]

In most programming languages this ‘paired’ data can be made into a list like this, it’s not as complex as json and can be stored in a type of variable called an array. Javascript handles all kinds of variable types, you just have to ‘declare’ what you want to call it and format it like this.

var c = [[0, 0], [1, 1], [2, 1], [3, 0], [4, 1], [4, 0]];

We’re going to use this as a data source to draw a shape in the OpenJSCAD browser by copying this bit of code and pasting into the code panel and using SHIFT & RETURN to render it.

You’ll notice that line 15 of that code are my graph plots. You could make your own graph and call the axes whatever you want and represent anything you want.

Drawing is mapping to

The exercise above maps data spatially into a shape you can 3D print, but of course we could do this with drawing just as well. We’ve prepared some tracing paper with some Graph paper printed on it and we’d like you to trace a single line over something and then draw x and y axes around them and plot this in a regular fashion. Use the grid to help and generate a list of numbers which are the spatial coordinates of the line you have drawn.

Then type this list to replace the var c list on line 15 and see what happens..

Why Data

Usually all the data stuff when we make digital images or video happens behind the scenes so you never think about it and for good reason; if you want to manipulate images in creative ways you often just want instant results. Then you can get on with the important stuff of making work.

Computer scientists have abstracted all the data stuff away so you can use it as a tool which is all good. However, thinking on a simple level of how a digital image is made up of individual ‘pixels’ with numbers assigned to it has it’s advantages. When images are turned into a list of numbers; into data with a very specific schema or format, they can be ‘translated’ and manipulated in powerful ways. Just thinking of an image like this gives you access to a literacy in what your tools are actually doing which could lead to you making your own tools or using your existing ones in new ways. And literacy can lead to the intuition and development we often need in our art practices.

In many ways a digital image is not an image at all but more like an abstract ‘spatial’ map of related coordinates and numbers in a colour space that use a format or schema that other people and machines know about so they can be shared on phones, projectors, monitors, or inkjet printers.

So our image data ‘maps’, can be read into other formats just as someone drawing will map visual experience to marks on paper; images can be turned into sound, manipulated as if they were so and translated back into images.

You can also make up data that’s not referring to sound or colour but perhaps record experiences or find data online.

This is a series of exercises to show that doing that is not that difficult.

Further Tools & Research

Data Sources

Stuck for spurious correlations in data? Huw showed me some here!

3D design tools toward 3D printing and fabrication

Topography Data

Want to print out .stl files from the topography on Google Maps? Have a look at Terrain2STL and try importing to tinkerCAD or OPENSCAD. Then mash up/combine with some anonomised personal data geometry you made previously. You can’t import with the vanilla OpenJSCAD but you can do if you build your own from OpenJSCAD on github. I’d recommend merging it in TinkerCAD or a standalone OpenSCAD application.

I’ve made a model of the topography around LICA here for you to look at

Going Further: Coding for Artists

Want to find out more about learning this kind of practice it’s well worth starting out with Processing. Initially check out Dan Shiffman’s (Processing inventor) Highly Enthusiastic video.

Then browse the work of a huge community of artists and creatives on the Open Processing site which lets you get coding straight away in the browser without downloading anything. You can then move onto downloading to your own computer.

Then you’ll build up knowledge and can go anywhere; if you want to make physical sensors to generate data it’s worth checking out artist Laura Pullig workshop tutorial called What Does Health Look Like using Processing and the BBC Microbit a £15 computer system for learning Physical computing.

Play

Want to just find more weird image generation tools without coding? Just browse around Open Processing and use the search for abstract things like ‘Circles’, ‘Squares’, ‘Spirals’ or even maths like ‘Fourier’

Just play with them, click on the </> button and change a few numbers see what happens: you dont have to learn Processing, just generate shapes screengrab, print or re-paint them; if you really like it and use some code in your work just refer back to it somehow just like a programmer will share who he modified or hacked code from. If you sign in with an openprocessing account you can follow the community members you like, or ‘fork’ your own version; this is just code-talk for modification; a fork in the road

Artists who use Data

Example established artistic practices that use data

Glitchers

Rosa Menkman “vernacular of file formats”

Nick Briz “glitch codec tutorial” video

Takeshi Murata “pink dot”

Phillip Sterns “glitch textiles”

Data Textiles and Painting

Dan Hays work hand takes the abstracted cartesian ‘image maps’ back to their material artefacts through hand painting pixels. Ironic that programmers spend years on the data architecture of images for convenience only for the image makers to un-convenience it almost in protest.

Sam Meech makes very interesting takes on very simple digital images and translates them with knitting machines, video, performance and interaction

Data Art & Activism

James Bridle is a journalist and artist uncovering the infrastructure of our now ubiquitous data culture.

Forensic Architecture famously won the 2018 Turner Prize and challenge what artists (or architects) are meant to do at all

Julian Oliver started out doing high end yet philosophically straightforward media-interaction work like Fijuu which I curated back in 2008, to developing the Critical Engineering Manifesto and making work and projects on the edge of speculative design and tools for network activism.

Lucy Kimbell Artist and researcher, writer and strategic design in social innovation. Considering and participating in wider data cultures can help sustain an artistic practice

Directory of data artists Big Bang Data Project at Somerset House

Ellie Harrison, a self confessed recovering data addict who has been recording how much she swims every day for years until she swims the atlantic. Also famously connected a vending machine to austerity data, campaigns to Bring Back British Rail and got roasted by the Daily Record for an Arts Council Funded Project for her to only work within the Greater Glasgow Strathclyde region to reduce her carbon footprint.

Sam Thulin does a Data Sensor Drone-By

Mo’ webgl Mo’ Problems

Since writing this I’ve noticed that with the recent auto updates of Chrome google have messed up webgl and have so far failed to do much about it.

A fix that worked for me was to enable webgl related flags by typing in these addresses into the address bar: chrome://flags/#ignore-gpu-blacklist and chrome://flags/#enable-webgl-draft-extensions

Then goto chrome://settings/ show Advanced Settings and under System enable the switch Use hardware acceleration when available so it’s on.

Failing that use Firefox. It’s nicer anyway.