Another Home Page Blog - wgethttps://blog.anotherhomepage.org/2008-03-23T14:55:00+01:00Créer un miroir CentOS pour les updates2008-03-23T14:55:00+01:002008-03-23T14:55:00+01:00Nils Ratuszniktag:blog.anotherhomepage.org,2008-03-23:/post/2008/03/23/Creer-un-miroir-CentOS-pour-les-updates/<p>Et tu sync, et tu sync, et tu wget</p>
<p>Hier, j'ai commencé à me dire que ça serait sympa de rendre encore plus rapide mes installations de machines virtuelles CentOS 5. Donc, avant d'attaquer le trifouillage de mes kickstarts afin d'appliquer mes configurations de manière automatique, j'en suis venu à …</p><p>Et tu sync, et tu sync, et tu wget</p>
<p>Hier, j'ai commencé à me dire que ça serait sympa de rendre encore plus rapide mes installations de machines virtuelles CentOS 5. Donc, avant d'attaquer le trifouillage de mes kickstarts afin d'appliquer mes configurations de manière automatique, j'en suis venu à la conclusion qu'avoir le dépôt des mises à jour en local serait plus rapide que de devoir tout récupérer sur Internet.</p>
<p>Donc, il me faut me synchroniser avec un miroir. Comme je suis chez Free, j'ai voulu utiliser leur miroir. Oui mais, ce n'est qu'un miroir FTP. Et la plupart des scripts de synchronisation de miroir utilisent <a href="http://rsync.samba.org/">rsync</a>. Je me suis donc rabattu sur le miroir de <a href="http://www.ipsl.jussieu.fr/">l'IPSL Jussieu</a> qui possède un mirroir <a href="http://distrib-coffee.ipsl.jussieu.fr/pub/linux/">http</a>, <a href="ftp://distrib-coffee.ipsl.jussieu.fr/pub/linux/">ftp</a> et <a href="rsync://distrib-coffee.ipsl.jussieu.fr/pub/linux/">rsync</a>. Génial ! Quelques instants plus tard, j'adapte <a href="http://lists.centos.org/pipermail/centos-mirror/2005-April/000936.html">ce script</a> qui donne ceci :</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1">#################################################################################</span>
<span class="c1"># #</span>
<span class="c1"># CentOS mirror syncer #</span>
<span class="c1"># Comes from : #</span>
<span class="c1"># #</span>
<span class="c1"># http://lists.centos.org/pipermail/centos-mirror/2005-April/000936.html #</span>
<span class="c1"># http://wiki.monserveurperso.com/wakka.php?wiki=MirroringFTP #</span>
<span class="c1"># man wget #</span>
<span class="c1"># #</span>
<span class="c1">#################################################################################</span>
<span class="nv">RSYNC</span><span class="o">=</span><span class="sb">`</span>which rsync<span class="sb">`</span>
<span class="nv">RSYNC_OPTS</span><span class="o">=</span><span class="s2">"-aHv --delete --bwlimit=512 "</span>
<span class="nv">CHOWN</span><span class="o">=</span><span class="sb">`</span>which chown<span class="sb">`</span>
<span class="nv">CHMOD</span><span class="o">=</span><span class="sb">`</span>which chmod<span class="sb">`</span>
<span class="nv">MIRROR</span><span class="o">=</span><span class="s2">"rsync://distrib-coffee.ipsl.jussieu.fr/pub/linux/centos/5/updates/i386/"</span>
<span class="nv">LOCAL</span><span class="o">=</span><span class="s2">"/home/ftp/mirrors/centos/5/updates/i386/"</span>
<span class="nv">PIDFILE</span><span class="o">=</span><span class="s2">"/var/run/centosmirror.pid"</span>
<span class="c1"># Don't run it twice...</span>
<span class="k">if</span> <span class="o">[</span> -f <span class="s2">"</span><span class="nv">$PIDFILE</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">RUNPID</span><span class="o">=</span><span class="sb">`</span>cat <span class="nv">$PIDFILE</span><span class="sb">`</span>
<span class="k">if</span> ps -p <span class="nv">$RUNPID</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">"CentOS Mirror is already running..."</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">"CentOS Mirror pid found but process dead, cleaning up"</span>
rm -f <span class="nv">$PIDFILE</span>
<span class="k">fi</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">"No CentOS Mirror Process Detected"</span>
<span class="k">fi</span>
<span class="nb">echo</span> <span class="nv">$$</span> > <span class="nv">$PIDFILE</span>
<span class="nb">echo</span> -n <span class="s2">"CentOS Mirror Started at "</span>
date
<span class="c1"># Download, set up rights</span>
<span class="nv">$RSYNC</span> <span class="nv">$RSYNC_OPTS</span> <span class="nv">$MIRROR</span> <span class="nv">$LOCAL</span>
<span class="nv">$CHOWN</span> -R apache:apache <span class="nv">$LOCAL</span>
<span class="nv">$CHMOD</span> -R <span class="m">755</span> <span class="nv">$LOCAL</span>
<span class="nb">echo</span> -n <span class="s2">"CentOS Mirror Ended at "</span>
date
rm -f <span class="nv">$PIDFILE</span>
</code></pre></div>
<p>Ca, c'était la version rsync. Et si on est derrière un proxy, et qu'on ne peut pas faire du rsync? Pour le moment je me suis concentré sur ftp, mais ce script devrait tourner pour http :</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1">#################################################################################</span>
<span class="c1"># #</span>
<span class="c1"># CentOS mirror syncer #</span>
<span class="c1"># Comes from : #</span>
<span class="c1"># #</span>
<span class="c1"># http://lists.centos.org/pipermail/centos-mirror/2005-April/000936.html #</span>
<span class="c1"># http://wiki.monserveurperso.com/wakka.php?wiki=MirroringFTP #</span>
<span class="c1"># man wget #</span>
<span class="c1"># #</span>
<span class="c1">#################################################################################</span>
<span class="c1"># Variables</span>
<span class="nv">WGET</span><span class="o">=</span><span class="sb">`</span>which wget<span class="sb">`</span>
<span class="nv">WGET_OPTS</span><span class="o">=</span><span class="s2">"-m --limit-rate=512k -nH --cut-dir 5"</span>
<span class="nv">CHOWN</span><span class="o">=</span><span class="sb">`</span>which chown<span class="sb">`</span>
<span class="nv">CHMOD</span><span class="o">=</span><span class="sb">`</span>which chmod<span class="sb">`</span>
<span class="nv">MIRROR</span><span class="o">=</span><span class="s2">"ftp://ftp.free.fr/mirrors/ftp.centos.org/5/updates/i386/*"</span>
<span class="nv">LOCAL</span><span class="o">=</span><span class="s2">"/home/ftp/mirrors/centos/5/updates/i386/"</span>
<span class="nv">PIDFILE</span><span class="o">=</span><span class="s2">"/var/run/centosmirror.pid"</span>
<span class="c1"># Don't run it twice...</span>
<span class="k">if</span> <span class="o">[</span> -f <span class="s2">"</span><span class="nv">$PIDFILE</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">RUNPID</span><span class="o">=</span><span class="sb">`</span>cat <span class="nv">$PIDFILE</span><span class="sb">`</span>
<span class="k">if</span> ps -p <span class="nv">$RUNPID</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">"CentOS Mirror is already running..."</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">"CentOS Mirror pid found but process dead, cleaning up"</span>
rm -f <span class="nv">$PIDFILE</span>
<span class="k">fi</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">"No CentOS Mirror Process Detected"</span>
<span class="k">fi</span>
<span class="nb">echo</span> <span class="nv">$$</span> > <span class="nv">$PIDFILE</span>
<span class="c1"># wget + ftp = .listing files</span>
find <span class="nv">$LOCAL</span> -name <span class="s2">".listing"</span> -delete
<span class="nb">echo</span> -n <span class="s2">"CentOS Mirror Started at "</span>
date
<span class="c1"># Download, set up rights</span>
<span class="nv">$WGET</span> <span class="nv">$WGET_OPTS</span> <span class="nv">$MIRROR</span> -P <span class="nv">$LOCAL</span>
<span class="nv">$CHOWN</span> -R apache:apache <span class="nv">$LOCAL</span>
<span class="nv">$CHMOD</span> -R <span class="m">755</span> <span class="nv">$LOCAL</span>
<span class="nb">echo</span> -n <span class="s2">"CentOS Mirror Ended at "</span>
date
rm -f <span class="nv">$PIDFILE</span>
find <span class="nv">$LOCAL</span> -name <span class="s2">".listing"</span> -delete
</code></pre></div>
<p>On remarquera que c'est grosso modo la même chose, la version wget générant des fichiers .listing pour le téléchargement récursif, je les efface avant et après coup, au cas où.Comme mon miroir est aussi dispo en http, je rend le tout accessible pour l'utilisateur apache. Ce dernier script reste cependant à améliorer, car contrairement à rsync, il n'efface pas les fichiers présents en local mais disparus du miroir distant.</p>