Animate your edit

A script and some simple Milky Way techniques this week

I've been doing a lot of Milky Way shooting recently and have developed some quick techniques for getting consistent results in Darktable. In this episode, I share my night photography processing tips and tricks. I also share with you my script to create a slideshow video from your sidecar XMP files. My script can accept multiple file inputs, so it can follow an edit through various saved points along the way.

This video is the latest in our faster format but packed with information. It builds on knowledge from previous videos, running full-tilt through tools and methods to achieve the goals. For fuller explanations, return to the Home Page and scroll down to Topics Discussed or Modules Used on the right-hand bar. Click on any of these, and we assemble a complete playlist of every instance from a knowledge base of our first 27 videos for quick review right here on the site.
Thank you for watching!

Milky Way RAW
Milky Way RAW XMP
Milky Way TIF
Milky Way TIF XMP
Milky Way final JPG
Milky Way slideshow of edit

Script to make slideshow from sidecar files

#!/bin/zsh

# usage: dt_movie.zsh file1.RAW file2.tif etc
# requires the use of sidecar xmp files
# make sure Darktable isn't running because only 1 instance is allowed at a time

frame_seconds=2.5
frame_width=1920
frame_height=1080
font_size=24
extra_frames=10
rm -fr /tmp/dt_movie*
mkdir /tmp/dt_movie &> /dev/null
typeset -Z4 index1 movie_index hist_index frame
index1=0; movie_index=1
first=$1
echo
echo "index /t hist /t enabled /t operation /t filename" |expand -t16
echo "---"
for file in "$@"; do
  index2=0
  grep -B1 'enabled=' $file.xmp |tr 'n' ' ' |sed -e 's/-- */!/g' |tr '!' 'n' |cut -d'"' -f2,4 |tr '"' ' ' |while read operation enabled; do
    echo "$index1 t $index2 t $enabled t $operation t $file" |expand -t16 >>/tmp/dt_movie.list
    ((index1++))
    ((index2++))
  done
done
((index1--))
cat /tmp/dt_movie.list
echo
echo -n "starting point (1): "
read starting_point
echo
[ -z $starting_point ] && starting_point=0
seq $starting_point $index1 |while read hist_index; do
  grep "^$hist_index" /tmp/dt_movie.list |read a b c d e
  if [ $c -eq 1 ]; then
    ((index2++))
    rm /tmp/dt_movie.jpg &> /dev/null
    cp $e.xmp /tmp/dt_movie.xmp
    sed -i "s/darktable:history_end.*$/darktable:history_end="$b">/" /tmp/dt_movie.xmp
    darktable-cli "$e" /tmp/dt_movie.xmp /tmp/dt_movie.jpg --width $frame_width --height $frame_height &> /dev/null
    echo "$a"/"$index1" $d |read text
    echo $text
    convert /tmp/dt_movie.jpg -pointsize $font_size -fill "#FFFFFF" -draw "text 10,32 '$text'" -background black -gravity center -extent "$frame_width"x"$frame_height" /tmp/dt_movie/$movie_index.jpg &> /dev/null
    ((movie_index++))
  fi
done
seq 1 $extra_frames |while read frame; do
  convert /tmp/dt_movie.jpg -pointsize $font_size -fill "#FFFFFF" -draw "text 10,32 '$text - finished'" -background black -gravity center -extent "$frame_width"x"$frame_height" /tmp/dt_movie/$movie_index.jpg &> /dev/null
  ((movie_index++))
  echo $frame
done
avconv -f image2 -r $((1/$frame_seconds)) -i /tmp/dt_movie/%04d.jpg -aspect 16:9 -b:v 15000k -y $first.avi &> /dev/null
rm -fr /tmp/dt_movie*

36 thoughts on “Animate your edit

  1. Hi Mouse, Harry

    That’s fine, I’ve got the script running with a few modifications. I just wanted to let you know about those little issues.

    Thank you both,
    Regards,
    Nicolás

  2. That was Mouse responding, cause I’ve been buried in work the past couple of days. Sorry for the confusion. I’ll have a look later today.

  3. Pingback: Weekly Edit 40: Animate your edit – darktable FR

  4. Dear Mouse and Harry,

    I mean the text on the dt_movie. The resultant .avi file.

    Thank you both for your time.
    Regards,
    Nicolás

  5. Harry,

    I forgot to mention that the overlay text is not synchronized with the right frame.

    e.g : When it says “Clipping”, the video is showing the “watermark” frame.

    Regards,
    Nicolás

    • Ah, yes – the transcription is an auto-transcribe feature from YouTube these days. In all of the earliest episodes, plus one or two more (the C-LUT one for sure), Mouse transcribed and synchronized the text so it was nearly perfectly accurate. But as our gallery took over our time, we’ve just let YouTube do its automatic text. Certainly not perfect, but they do surprisingly well for the most part.
      –Mouse and Harry

  6. Hi Harry

    1)
    [export_job] exported to `/tmp/dt_movieIMG_8478.jpg’
    0000/0003 watermark
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0001.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    rm: cannot remove ‘/tmp/dt_movie.jpg’: No such file or directory
    [export_job] exported to `/tmp/dt_movieIMG_8478_01.jpg’

    So I replaced /tmp/dt_movie.jpg for /tmp/dt_movie_.jpg (note the underscore) . After this change the script started to work.

    2) It seems that the script is not processing the last module of the list.
    Let’s say that the .xmp has 4 modules to be applied

    watermark
    clipping
    tonecurve
    colorcorrection

    The script is processing watermark, clipping and tonecurve but is missing the colorcorrection module.

    The xmp :
    watermark hist = 1
    clipping hist = 2
    tonecurve hist = 3
    colorcorrection hist = 4

    In the dt_movie.list you have

    0000 0 1 watermark IMG_8478.CR2
    0001 1 1 clipping IMG_8478.CR2
    0002 2 1 tonecurve IMG_8478.CR2
    0003 3 1 colorcorrection IMG_8478.CR2

    Hist number 3 (colorcorrection) is preventing the hist 4 of the original .xmp.

    Thank you for sharing! Saved me from using screenshot !!!
    Regards,
    Nicolás

  7. The script is exporting a file called dt_movieIMG_8563_03.jpg
    Then is asking for dt_movie.jpg’
    is this ok ?

    [export_job] exported to `dt_movieIMG_8563_03.jpg’
    0028/0028 shadhi
    convert: unable to open image `dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.

  8. nicolas@osiris:/tmp$ darktable-cli
    usage: darktable-cli [] [–width ,–height ,–bpp ,–hq ,–upscale ,–verbose] [–core ]
    nicolas@osiris:/tmp$

  9. I suppose selecting 28 as “starting point” shouldn’t be an issue ..

    Result:
    ————————————————————-

    starting point (1): 28

    [export_job] exported to `/tmp/dt_movieIMG_8563.jpg’
    0028/0028 shadhi
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0001.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0002.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0001
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0003.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0002
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0004.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0003
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0005.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0004
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0006.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0005
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0007.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0006
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0008.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0007
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0009.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0008
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0010.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0009
    convert: unable to open image `/tmp/dt_movie.jpg’: No such file or directory @ error/blob.c/OpenBlob/2712.
    convert: no images defined `/tmp/dt_movie/0011.jpg’ @ error/convert.c/ConvertImageCommand/3210.
    0010
    ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
    built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
    configuration: –prefix=/usr –extra-version=0ubuntu0.16.04.1 –build-suffix=-ffmpeg –toolchain=hardened –libdir=/usr/lib/x86_64-linux-gnu –incdir=/usr/include/x86_64-linux-gnu –cc=cc –cxx=g++ –enable-gpl –enable-shared –disable-stripping –disable-decoder=libopenjpeg –disable-decoder=libschroedinger –enable-avresample –enable-avisynth –enable-gnutls –enable-ladspa –enable-libass –enable-libbluray –enable-libbs2b –enable-libcaca –enable-libcdio –enable-libflite –enable-libfontconfig –enable-libfreetype –enable-libfribidi –enable-libgme –enable-libgsm –enable-libmodplug –enable-libmp3lame –enable-libopenjpeg –enable-libopus –enable-libpulse –enable-librtmp –enable-libschroedinger –enable-libshine –enable-libsnappy –enable-libsoxr –enable-libspeex –enable-libssh –enable-libtheora –enable-libtwolame –enable-libvorbis –enable-libvpx –enable-libwavpack –enable-libwebp –enable-libx265 –enable-libxvid –enable-libzvbi –enable-openal –enable-opengl –enable-x11grab –enable-libdc1394 –enable-libiec61883 –enable-libzmq –enable-frei0r –enable-libx264 –enable-libopencv
    WARNING: library configuration mismatch
    avcodec configuration: –prefix=/usr –extra-version=0ubuntu0.16.04.1 –build-suffix=-ffmpeg –toolchain=hardened –libdir=/usr/lib/x86_64-linux-gnu –incdir=/usr/include/x86_64-linux-gnu –cc=cc –cxx=g++ –enable-gpl –enable-shared –disable-stripping –disable-decoder=libopenjpeg –disable-decoder=libschroedinger –enable-avresample –enable-avisynth –enable-gnutls –enable-ladspa –enable-libass –enable-libbluray –enable-libbs2b –enable-libcaca –enable-libcdio –enable-libflite –enable-libfontconfig –enable-libfreetype –enable-libfribidi –enable-libgme –enable-libgsm –enable-libmodplug –enable-libmp3lame –enable-libopenjpeg –enable-libopus –enable-libpulse –enable-librtmp –enable-libschroedinger –enable-libshine –enable-libsnappy –enable-libsoxr –enable-libspeex –enable-libssh –enable-libtheora –enable-libtwolame –enable-libvorbis –enable-libvpx –enable-libwavpack –enable-libwebp –enable-libx265 –enable-libxvid –enable-libzvbi –enable-openal –enable-opengl –enable-x11grab –enable-libdc1394 –enable-libiec61883 –enable-libzmq –enable-frei0r –enable-libx264 –enable-libopencv –enable-version3 –disable-doc –disable-programs –disable-avdevice –disable-avfilter –disable-avformat –disable-avresample –disable-postproc –disable-swscale –enable-libopencore_amrnb –enable-libopencore_amrwb –enable-libvo_aacenc –enable-libvo_amrwbenc
    libavutil 54. 31.100 / 54. 31.100
    libavcodec 56. 60.100 / 56. 60.100
    libavformat 56. 40.101 / 56. 40.101
    libavdevice 56. 4.100 / 56. 4.100
    libavfilter 5. 40.101 / 5. 40.101
    libavresample 2. 1. 0 / 2. 1. 0
    libswscale 3. 1.101 / 3. 1.101
    libswresample 1. 2.101 / 1. 2.101
    libpostproc 53. 3.100 / 53. 3.100
    [image2 @ 0x1c89ea0] Could find no file with path ‘/tmp/dt_movie/%04d.jpg’ and index in the range 0-4
    /tmp/dt_movie/%04d.jpg: No such file or directory
    nicolas@osiris:~/Herramientas/mycommands/dt_movie$

  10. LoL
    sudo apt-get install libav-tools

    Convert it was already installed .

    I’ve select 20 as “starting point” , but not resultant file was found … let me try the whole process again from Starting point : 1 …. just in case..

  11. Well, the process finished , but I can’t see any resultant file.

    you sure “rm -rf /tmp/dt_movie” is mandatory? Or the resultant file should be found where I run the script?

    This is the output

    nicolas@osiris:~/Herramientas/mycommands/dt_movie$ ./dt_movie.zsh IMG_8563.CR2

    index hist enabled operation filename

    0000 0 1 monochrome IMG_8563.CR2
    0001 1 1 colisa IMG_8563.CR2
    0002 2 1 colisa IMG_8563.CR2
    0003 3 1 monochrome IMG_8563.CR2
    0004 4 1 highpass IMG_8563.CR2
    0005 5 1 highpass IMG_8563.CR2
    0006 6 0 bilat IMG_8563.CR2
    0007 7 1 tonecurve IMG_8563.CR2
    0008 8 1 colisa IMG_8563.CR2
    0009 9 0 colisa IMG_8563.CR2
    0010 10 1 colisa IMG_8563.CR2
    0011 11 1 shadhi IMG_8563.CR2
    0012 12 1 shadhi IMG_8563.CR2
    0013 13 1 colorcorrection IMG_8563.CR2
    0014 14 1 clipping IMG_8563.CR2
    0015 15 1 tonecurve IMG_8563.CR2
    0016 16 1 tonecurve IMG_8563.CR2
    0017 17 1 tonecurve IMG_8563.CR2
    0018 18 1 tonecurve IMG_8563.CR2
    0019 19 1 tonecurve IMG_8563.CR2
    0020 20 1 monochrome IMG_8563.CR2
    0021 21 1 clipping IMG_8563.CR2
    0022 22 1 defringe IMG_8563.CR2
    0023 23 1 denoiseprofile IMG_8563.CR2
    0024 24 1 clipping IMG_8563.CR2
    0025 25 1 monochrome IMG_8563.CR2
    0026 26 1 shadhi IMG_8563.CR2
    0027 27 1 shadhi IMG_8563.CR2
    0028 28 1 shadhi IMG_8563.CR2

    starting point (1):

    0000/0028 monochrome
    0001/0028 colisa
    0002/0028 colisa
    0003/0028 monochrome
    0004/0028 highpass
    0005/0028 highpass
    0007/0028 tonecurve
    0008/0028 colisa
    0010/0028 colisa
    0011/0028 shadhi
    0012/0028 shadhi
    0013/0028 colorcorrection
    0014/0028 clipping
    0015/0028 tonecurve
    0016/0028 tonecurve
    0017/0028 tonecurve
    0018/0028 tonecurve
    0019/0028 tonecurve
    0020/0028 monochrome
    0021/0028 clipping
    0022/0028 defringe
    0023/0028 denoiseprofile
    0024/0028 clipping
    0025/0028 monochrome
    0026/0028 shadhi
    0027/0028 shadhi
    0028/0028 shadhi
    0001
    0002
    0003
    0004
    0005
    0006
    0007
    0008
    0009
    0010
    nicolas@osiris:~/Herramientas/mycommands/dt_movie$

  12. it is still processing.

    You should compare the original file and the one you uploaded on wordpress , with and hexa tool. Or maybe using some editor with the “view all symbols” enabled.

  13. yes,

    For some reason the “post box” didn’t show both lines.

    $ diff dt_movie.zsh dt_movie.zsh.old
    echo “index /t hist /t enabled /t operation /t filename” |expand -t16
    echo “index t hist t enabled t operation t filename” |expand -t16

  14. $ diff dt_movie.zsh dt_movie.zsh.old
    18c18
    echo “index t hist t enabled t operation t filename” |expand -t16

  15. Sorry, same result.

    Please, could you send me you script as a .tar or .zip file? I am using sublimetext and gedit to create the file. I would like to try the original script file.

  16. WordPress had stripped the backslashes off my echo. It is repaired. Please try the new version. Thanks for helping me.

  17. hmm, zsh is already installed.

    nicolas@osiris:/mycommands$ ./dt_movie.zsh IMG_8563.CR2

    or
    osiris% ./dt_movie.zsh IMG_8563.CR2

    I will dig into the script a bit more .

  18. Okay, found it. Install zsh on your computer. It’s the shell I use and it parses the output differently.

  19. Yes, maybe the script is not able to find an item within the .xmp file, probably due to the DT version.
    I am running DT 2.2.4

    I’ve sent you the file,
    Thank you.

  20. Hmm ..
    cp: cannot stat ‘t 1 t IMG_8563.CR2.xmp’: No such file or directory
    sed: can’t read dt_movie.xmp: No such file or directory

    Part of my dt_movie.list:

    0000 t 0 t ochrome t mo t IMG_8563.CR2
    0001 t 1 t t 1 t IMG_8563.CR2
    0002 t 2 t t n t IMG_8563.CR2
    0003 t 3 t t colisa t IMG_8563.CR2
    0004 t 4 t t 1 t IMG_8563.CR2
    0005 t 5 t t n t IMG_8563.CR2
    0006 t 6 t t colisa t IMG_8563.CR2
    0007 t 7 t t 1 t IMG_8563.CR2
    0008 t 8 t t n t IMG_8563.CR2
    0009 t 9 t ochrome t mo t IMG_8563.CR2
    0010 t 10 t t 1 t IMG_8563.CR2
    0011 t 11 t t n t IMG_8563.CR2
    0012 t 12 t t highpass t IMG_8563.CR2
    0013 t 13 t t 1 t IMG_8563.CR2
    0014 t 14 t t n t IMG_8563.CR2

    The folder in which I am running the script contains two files:
    IMG_8563.CR2 and IMG_8563.CR2.xmp

    Not a permission issue..

Leave a Reply

Your email address will not be published.