Here is another version, without overlap between photos: https://twitter.com/juanbuhler/status/1379201237187395584 https://pbs.twimg.com/media/EyPn2IVUYAMv5BO?format=jpg&name=4096x4096 j -- Juan Buhler - http://www.juanbuhler.com On Sat, Apr 3, 2021 at 2:22 PM Juan Buhler <juanbuhler@gmail.com> wrote: > > I made a plot of about 3000 of my photos (all posted to my photoblog over > the years) according to positions in the plane that come from a neural > network. > > Without getting into technical detail: images that are close to each other > are semantically similar to each other. So there are areas of photos with > dogs, others with bicycles, on the beach, etc etc. > > https://twitter.com/juanbuhler/status/1378455676444270593 > > To see the high res image and zoom in, look at the file directly: > > https://pbs.twimg.com/media/EyFA3LlU4Ag0QRQ?format=jpg&name=4096x4096 > > I thought it was an interesting way of seeing a collection of photos and > discovering emerging visual themes. Also it's what I do for a living so I > figured why not. > > Most of these were taken on Pentax cameras, so on topic! :) > > j > > -- > Juan Buhler - http://www.juanbuhler.com >

Hello Juan, > On 4 Apr 2021, at 18:46, Juan Buhler <juanbuhler@gmail.com> wrote: > >> Just out of curiosity, what was the input to the Neural Network? >> Keywords, descriptions? > > No, it's just the pixels themselves. I'm using a pretrained convolutional > neural network, or CNN. You know how Google Photos is able to separate > photos into categories, with dogs, food, mountains, etc? That is done with > a neural network of the same type. > > These CNNs will output the confidence they have that an image belongs to > one of many classes they were trained for. But in the process they compute > a vector that sort of encodes what "features" exist in the image. Features > are things like lines, dots, patterns, and also combinations of things that > might form "higher level features", like eyes, bicycle wheels, etc etc. > These vectors are of very high dimension, in this case 2048. > > It turns out that points in this 2048-D space will be close to each other > if the images they come from are similar to each other. > > The process I'm using computes and saves this vector for each image. That > alone allows me to do image similarity search, by comparing these vectors. > > In order to make the plot, I use a technique that "folds" those 2048 > dimensions into two, so I can find a position for each image on the plane. > Okay, interesting, thanks for explaining! I have been doing something remotely similar with face-recognition on multiple live video-streams a few years ago, for supermarket customer-following systems ... > Hopefully I succeeded in making that explanation not too technical? > Not at all, I am a programmer too, mainly making disk and filesystem recovery software, in plain ‘C’ ;-) Regards, JvW ------------------------------------------------------------------ Jan van Wijk; https://www.dfsee.com/gallery

It all sounds pretty straightforward, but I'm curious about the last step of folding 2048D -> 2D. Must involve some kind of clustering? How is this done? This seems key. --gonz On Sun, Apr 4, 2021 at 11:47 AM Juan Buhler <juanbuhler@gmail.com> wrote: > > > Just out of curiosity, what was the input to the Neural Network? > > Keywords, descriptions? > > No, it's just the pixels themselves. I'm using a pretrained convolutional > neural network, or CNN. You know how Google Photos is able to separate > photos into categories, with dogs, food, mountains, etc? That is done with > a neural network of the same type. > > These CNNs will output the confidence they have that an image belongs to > one of many classes they were trained for. But in the process they compute > a vector that sort of encodes what "features" exist in the image. Features > are things like lines, dots, patterns, and also combinations of things that > might form "higher level features", like eyes, bicycle wheels, etc etc. > These vectors are of very high dimension, in this case 2048. > > It turns out that points in this 2048-D space will be close to each other > if the images they come from are similar to each other. > > The process I'm using computes and saves this vector for each image. That > alone allows me to do image similarity search, by comparing these vectors. > > In order to make the plot, I use a technique that "folds" those 2048 > dimensions into two, so I can find a position for each image on the plane. > > Hopefully I succeeded in making that explanation not too technical? > > j > > -- > Juan Buhler - http://www.juanbuhler.com > > > On Sun, Apr 4, 2021 at 1:53 AM <pentax@dfsee.com> wrote: > > > Interesting Juan! > > > > > > > On 3 Apr 2021, at 23:22, Juan Buhler <juanbuhler@gmail.com> wrote: > > > > > > I made a plot of about 3000 of my photos (all posted to my photoblog over > > > the years) according to positions in the plane that come from a neural > > > network. > > > > > > Without getting into technical detail: images that are close to each > > other > > > are semantically similar to each other. So there are areas of photos with > > > dogs, others with bicycles, on the beach, etc etc. > > > > > > https://twitter.com/juanbuhler/status/1378455676444270593 > > > > > > To see the high res image and zoom in, look at the file directly: > > > > > > https://pbs.twimg.com/media/EyFA3LlU4Ag0QRQ?format=jpg&name=4096x4096 > > > > > > > Just out of curiosity, what was the input to the Neural Network? > > > > Keywords, descriptions? > > > > > > > I thought it was an interesting way of seeing a collection of photos and > > > discovering emerging visual themes. Also it's what I do for a living so I > > > figured why not. > > > > > > > Indeed, touching on ‘big data’ ;-) > > > > > > Regards, JvW > > > > ------------------------------------------------------------------ > > Jan van Wijk; https://www.dfsee.com/gallery > > > > -- > > %(real_name)s Pentax-Discuss Mail List > > To unsubscribe send an email to pdml-leave@pdml.net > > to UNSUBSCRIBE from the PDML, please visit the link directly above and > > follow the directions. > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-leave@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions. -- "I think my ex-wife has weekly lessions with the Devil on how to be more evil. I don't know how much she charges him." - Emo Philips

Gonz, In order to go from 2048D to 2D, there are a couple of techniques for dimensionality reduction out there. I'm using the most common one. Look here: https://lvdmaaten.github.io/tsne/ j -- Juan Buhler - http://www.juanbuhler.com

Bookmarked! Thank you Juan. --Gonz On Wed, Apr 14, 2021 at 4:04 PM Juan Buhler <juanbuhler@gmail.com> wrote: > > Gonz, > > In order to go from 2048D to 2D, there are a couple of techniques for > dimensionality reduction out there. I'm using the most common one. Look > here: > > https://lvdmaaten.github.io/tsne/ > > j > > -- > Juan Buhler - http://www.juanbuhler.com > -- > %(real_name)s Pentax-Discuss Mail List > To unsubscribe send an email to pdml-leave@pdml.net > to UNSUBSCRIBE from the PDML, please visit the link directly above and follow the directions. -- "I think my ex-wife has weekly lessions with the Devil on how to be more evil. I don't know how much she charges him." - Emo Philips