Possibly useful tool for shoops

[The Clark]

www.ebsynth.com/

I can think of a few ways this could be helpful when doing shoops. I made this test gif to see if I could use it for faceswapping with just one frame of my face on Roy Batty. The more keyframes you use, the better your results can be.

Tutorial:

 

[bigwaverider]

www.ebsynth.com/

I can think of a few ways this could be helpful when doing shoops. I made this test gif to see if I could use it for faceswapping with just one frame of my face on Roy Batty. The more keyframes you use, the better your results can be.

Tutorial:

Way cool stuff brah. I can see @Arqueto making the 1337 shoop soon. He is learning to draw and paint. I need to do my Rogan one one day myself. Already got this stuff cut out.

Seems like it is gonna take a real long time to do one of those properly. Maybe you can use some Photoshop filters instead of painting it?

 

[Arqueto]

Very cool. I will take a look at this.

 

[The Clark]

Seems like it is gonna take a real long time to do one of those properly. Maybe you can use some Photoshop filters instead of painting it?

From what I can tell, I think it basically works like video compression does. Like how videos store information about how pixels change relative to keyframes, rather than storing each frame as its own still image. You can basically use whatever image you want, and those pixels will move and change in the same way the corresponding pixels in the video do, if that makes sense. So, I believe you could do something like drawing a mask/matte for someone on one frame and let this draw it over the rest. I'm going to mess around with it more when I get home.

 

[bigwaverider]

From what I can tell, I think it basically works like video compression does. Like how videos store information about how pixels change relative to keyframes, rather than storing each frame as its own still image. You can basically use whatever image you want, and those pixels will move and change in the same way the corresponding pixels in the video do, if that makes sense. So, I believe you could do something like drawing a mask/matte for someone on one frame and let this draw it over the rest. I'm going to mess around with it more when I get home.

I doubt the stuff in red is accurate.

 

[Arqueto]

How long did it take you to do that clip @The Clark?

 

[The Clark]

How long did it take you to do that clip @The Clark?

It just took a couple of minutes to split the video into frames and make the folders, then a couple minutes to process/render. Probably depends on your computer, and I was using a laptop that's not too beefy. The most time consuming part was just shooping my face on, which didn't take long, either. The first frame of the gif is the only thing that I did manually. If I was going to make the clip longer, I would probably do another key frame right about where this one ended and his mouth opens. I'm not sure how useful this will be right now, but I'm interested in testing it out and exploring different possibilities.

 

[Arqueto]

It's really cool how it can pick up on the mouth movement and apply it like that, I've watched some of the deepfake stuff and that looks like a lot of work, but this seems really simple. I might have a play around with it when I get a chance, seems like we could do some good stuff for shooping with this.

 

[The Clark]

I doubt the stuff in red is accurate.

Well, I guess I was being a bit optimistic there, lol. But, I tried it out, and you know what? It actually did okay! It's kind of silly, because you would still need to convert the images to some type of mask and clean them up, but I could see it being useful sometime. Here are two clips I did. All I did was paint Max green on one frame and feed it through:

But the real fun seems like doing stuff like this:

All I did was edit one frame. That blows my mind! When I'm off work, I'm going to try to make some nice ones and really spend some time on it.

It's really cool how it can pick up on the mouth movement and apply it like that, I've watched some of the deepfake stuff and that looks like a lot of work, but this seems really simple. I might have a play around with it when I get a chance, seems like we could do some good stuff for shooping with this.

Yeah, this is like the lazy workaround version, lol. But it seems like you can pull off some really cool stuff with it. Let me know if you try it out!

 

[bigwaverider]

Well, I guess I was being a bit optimistic there, lol. But, I tried it out, and you know what? It actually did okay! It's kind of silly, because you would still need to convert the images to some type of mask and clean them up, but I could see it being useful sometime. Here are two clips I did. All I did was paint Max green on one frame and feed it through:

But the real fun seems like doing stuff like this:

All I did was edit one frame. That blows my mind! When I'm off work, I'm going to try to make some nice ones and really spend some time on it.



Yeah, this is like the lazy workaround version, lol. But it seems like you can pull off some really cool stuff with it. Let me know if you try it out!

That spiderman is really cool! did you draw a frame of it? did you pre mask it all?

I think ppl should also know that Photoshop can export png image sequences rather easily.

Go to "File > Export > Render Video" and select "Photoshop Image Sequence" from the mid pull down menu. Then you can select PNG as file format and change the name of the files.

The rest of this is geek stuff and may confuse a lot of people or it may help some.

It's a bit tricky to export a mask sequence and probably will have to be done frame by frame. I am thinking though.... There is a cheesey trick that I just thought of that may work in PS where you can export a mask as a image sequences.

1. Cut your person from your video sequence
2. Export that as as Quicktime File using "Alppha Channel" set to "Straight Unmatted". I mean u could do a .png sequence and it would be the same thing, but using QT video would be much easier and create one file instead of many.
3. Open that video in PS. I don't think opening it as a video file in the video timeline or importing it as a frame animation would matter, but opening it as video will be easier. If you are using a png sequence, put them all in one folder. We will now have a video of the cut out person on a clear background.
4. Make sure there is a black background.
5. Clip a Hue/Saturation Adjustment Layer to the video layer or the folder with the .png's. Make sure the "Lightness" slider is slid all the way to the right. This will make the cut out person all white in every frame. Since the person is now all white, and the background is black, VOILA! we have an Alpha Mask ready for export as a .png sequence.

 

[bigwaverider]

Well, I guess I was being a bit optimistic there, lol. But, I tried it out, and you know what? It actually did okay! It's kind of silly, because you would still need to convert the images to some type of mask and clean them up, but I could see it being useful sometime. Here are two clips I did. All I did was paint Max green on one frame and feed it through:

But the real fun seems like doing stuff like this:

All I did was edit one frame. That blows my mind! When I'm off work, I'm going to try to make some nice ones and really spend some time on it.



Yeah, this is like the lazy workaround version, lol. But it seems like you can pull off some really cool stuff with it. Let me know if you try it out!

As for your video AE has a "rotoscoping tool" and it does about as good of a job as your green mask, but you can fix the inaccuracies as you go along. It's pretty much the same thing as the "quick select tool" in PS. However, as you go along, if the selection gets inaccurate you can dial it in and continue.

 

[bigwaverider]

When I'm off work, I'm going to try to make some nice ones and really spend some time on it.

I will probably try it at some point. Looks really interesting. The potential is there for some cool Ideas. However I got some projects that need to be finished and ISL is coming up in one week. Too busy to do anything now.

 

[The Clark]

That spiderman is really cool! did you draw a frame of it? did you pre mask it all?

All I did was make this one frame near the end, and EBSynth did the rest. No masking at all! (I made that mask and didn't even use it, lol.)

It's a bit tricky to export a mask sequence and probably will have to be done frame by frame. I am thinking though.... There is a cheesey trick that I just thought of that may work in PS where you can export a mask as a image sequences.

I'm used to doing everything frame by frame, anyway, lol. I am going to keep playing around and see if I can get some better results. I'm honestly surprised how well it followed Max.

 

[bigwaverider]

All I did was make this one frame near the end, and EBSynth did the rest. No masking at all! (I made that mask and didn't even use it, lol.)

I'm used to doing everything frame by frame, anyway, lol. I am going to keep playing around and see if I can get some better results. I'm honestly surprised how well it followed Max.

ahh i see. u just copy/pasted parts of Spiderman on top of him. yea amazing that it could detect his left arm and hand.

 

[bigwaverider]

OK I tried one. I put Novinsky over Dana. It really distorted Novinsky's glasses so I had to cut them out and try to place them on top. Didnt want to make a huge project out if this, so this is how it came out. Also a little bug. I kept getting errors about it not finding the Keyframe image. It was driving me nuts. I had copy/pasted the file numbers into the two "stop boxes" and the "keyframe box" so out of desperation I manually typed them in and it worked. That took me a solid 45 minutes to figure out as I was trying so many different things to fix it. It added a lot of dither to Novinsky and I didn't want to take the time to fix it which probably would be to add dither to the rest of the image.

@Arqueto @The Clark

 

[bigwaverider]

This is Novinsky's head isolated. This is before i added the new glasses.

 

[Arqueto]

OK I tried one. I put Novinsky over Dana. It really distorted Novinsky's glasses so I had to cut them out and try to place them on top. Didnt want to make a huge project out if this, so this is how it came out. Also a little bug. I kept getting errors about it not finding the Keyframe image. It was driving me nuts. I had copy/pasted the file numbers into the two "stop boxes" and the "keyframe box" so out of desperation I manually typed them in and it worked. That took me a solid 45 minutes to figure out as I was trying so many different things to fix it. It added a lot of dither to Novinsky and I didn't want to take the time to fix it which probably would be to add dither to the rest of the image.

@Arqueto @The Clark

It's cool that it can do that from one frame but that looks a bit goofy there is a lot of weird stuff going on.

 

[bigwaverider]

It's cool that it can do that from one frame but that looks a bit goofy there is a lot of weird stuff going on.

yea. its not really made to animate heads though. we are already hacking the program.

 

[bigwaverider]

It's cool that it can do that from one frame but that looks a bit goofy there is a lot of weird stuff going on.

if I could make it look more like novisnky, it would still be cool enough for some shoops on sherdog. i think my source image for the template "keyframe" was just too shitty. And it had sunglasses which freaked the program out.

 

[Arqueto]

yea. its not really made to animate heads though. we are already hacking the program.

I'm really surprised at how well it follows the mouth movements.