稳定的扩散 - 视频
在Colab中自己尝试：
示例- “蓝莓意大利面”和“草莓意大利面”之间的变形
berry_good_spaghetti.2.mp4
安装
pip install stable_diffusion_videos用法
查看示例文件夹，例如脚本?
制作视频
注意：对于Apple M1体系结构，请改用Torch.float32，因为MPS上没有Torch.float16。
512. Multiples of 8 if < 512.
width=512, # use multiples of 64 if > 512. Multiples of 8 if < 512.
output_dir='dreams', # Where images/videos will be saved
name='animals_test', # Subdirectory of output_dir where images/videos will be saved
guidance_scale=8.5, # Higher adheres to prompt more, lower lets model take the wheel
num_inference_steps=50, # Number of diffusion steps per image generated. 50 is good default
)">
from stable_diffusion_videos import StableDiffusionWalkPipeline import torch pipeline = StableDiffusionWalkPipeline . from_pretrained ( "CompVis/stable-diffusion-v1-4" , torch_dtype = torch . float16 , ). to ( "cuda" ) video_path = pipeline . walk ( prompts = [ 'a cat' , 'a dog' ], seeds = [ 42 , 1337 ], num_interpolation_steps = 3 , height = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512. width = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512. output_dir = 'dreams' , # Where images/videos will be saved name = 'animals_test' , # Subdirectory of output_dir where images/videos will be saved guidance_scale = 8.5 , # Higher adheres to prompt more, lower lets model take the wheel num_inference_steps = 50 , # Number of diffusion steps per image generated. 50 is good default )
制作音乐视频
新的！可以通过提供音频文件的路径来添加音乐。音频将告知插值率，因此视频转移到节拍?
512. Multiples of 8 if < 512.
width=512, # use multiples of 64 if > 512. Multiples of 8 if < 512.
output_dir='dreams', # Where images/videos will be saved
guidance_scale=7.5, # Higher adheres to prompt more, lower lets model take the wheel
num_inference_steps=50, # Number of diffusion steps per image generated. 50 is good default
)">
from stable_diffusion_videos import StableDiffusionWalkPipeline import torch pipeline = StableDiffusionWalkPipeline . from_pretrained ( "CompVis/stable-diffusion-v1-4" , torch_dtype = torch . float16 , ). to ( "cuda" ) # Seconds in the song. audio_offsets = [ 146 , 148 ] # [Start, end] fps = 30 # Use lower values for testing (5 or 10), higher values for better quality (30 or 60) # Convert seconds to frames num_interpolation_steps = [( b - a ) * fps for a , b in zip ( audio_offsets , audio_offsets [ 1 :])] video_path = pipeline . walk ( prompts = [ 'a cat' , 'a dog' ], seeds = [ 42 , 1337 ], num_interpolation_steps = num_interpolation_steps , audio_filepath = 'audio.mp3' , audio_start_sec = audio_offsets [ 0 ], fps = fps , height = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512. width = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512. output_dir = 'dreams' , # Where images/videos will be saved guidance_scale = 7.5 , # Higher adheres to prompt more, lower lets model take the wheel num_inference_steps = 50 , # Number of diffusion steps per image generated. 50 is good default )
使用UI
from stable_diffusion_videos import StableDiffusionWalkPipeline , Interface
import torch
pipeline = StableDiffusionWalkPipeline . from_pretrained (
"CompVis/stable-diffusion-v1-4" ,
torch_dtype = torch . float16 ,
). to ( "cuda" )
interface = Interface ( pipeline )
interface . launch ()学分
这项工作是由@karpathy共享的脚本构建的。将脚本修改为此要旨，然后将其更新/修改为此回购。
贡献
您可以在此处提交任何问题/功能请求
享受?
下载源码
通过命令行克隆项目:
git clone https://github.com/nateraw/stable-diffusion-videos.git