My thoughts on Slackware, life and everything

taper.alienbase.nl mirror will lose rsync access

For the sixth time in just 5 days I had do a system_reset on my virtual machine which runs “taper.alienbase.nl” as well as “docs.slackware.com“. The virtual machine is crashing under the load that is put on it by demanding rsync processes. According to my pal who donated the use of this VM to me for free, the rsync download rate is at a continuous 100 Mbit/sec for most of the time. This is apparently too much for the server, as well as for my pal who had not anticipated this kind of bandwidth consumption. He has been paying quite a bit of extra money for the excess bandwidth during the past months.

I do not like it when someone loses money because of me, so drastic measures are required.

As of tomorrow (monday 15 Feb 2016) I will kill the rsync access to taper.alienbase.nl.

HTTP access will still be allowed, so people who configured slackpkg to download from my server will still be served.

I hope that this will put my pal at ease again, and also prevent the server from crashing.

Apologies for the inconvenience this causes to people who use rsync to keep their local mirrors up to date. I will try to find another location to host my SlackBuild repository as well as the multilib and KDE (ktown) repositories. The downtime of this VM caused downtime for docs.slackware.com as well which I think is unacceptible.

Eric

19 Comments

  1. John Yost

    Greetings Eric
    Guilty as charged.
    My apologies and I will in the future try to be a better site user.
    Thanks For All Your Efforts For Slackware
    John

  2. Mike Langdon

    Hi Eric,

    I have switched to your uk mirror for awhile. However, the checksums failed for the slackbuilds repo.

    Thanks for providing these builds – it would take me forever to build some of these. It is time for me to upgrade this ancient box.

  3. Mike Langdon

    Sorry – the failed part read:
    fails to download CHECKSUMS.md5

  4. Niki Kovacs

    Why not consider a Dedibox SC offer from online.net? 5,99 euros / month with unlimited bandwidth, it’s not a VPS but a *real* server, and Slackware runs great on it. I have three of these boxes (one bigger “classic” for production for 30 euros / month, two “SC” for backups and testing). Quality of service is great.

  5. alienbob

    Niki, that is indeed a nice price for something that does not need a lot of CPU power but lots of bandwidth.
    Still, another bill to pay.

  6. Karl

    I’ve got a couple of machines with similar bandwidth available (100Mbps each) which I can use to provide rsync mirrors. We don’t pay for traffic, so it won’t cost us anything extra. If you’re interested, I could have them set up by the end of the week. Let me know via email (let me know where I should pull from)

  7. alienbob

    Hi Karl

    Your mirrors is certainly welcome. If you have a mirror URL and decent bandwidth and a good uptime, I can add your URL to the list of mirrors that I mention every time when I do a repository update.
    I still allow rsync access for “slackware.uk” and that site is able to keep an uptodate mirror of my “taper.alienbase.nl” server. Any content you need to mirror can be obtained from that site. Yesterday, a mirror of the slackware-live ISOs was added to slackware.uk, and that is still syncingf. Hopefully the sync will be complete today. Then a mirror of all my stuff will be available on slackware.uk.

    Grateful for every reliable and fast mirror that people setup! Thanks.

  8. schu

    Eric, I can spool up a vm to mirror stuff in Seattle. Send me an email with an rsync url to source from, the frequency you would like updates, and I’ll get it built in the next day or two and return the hostname.

  9. alienbob

    At the moment the most accessible and also fast location would be slackware.uk. Darren who maintains that server has whitelist access to taper.alienbase.nl so he’ll never be far behind my updates.

    The most important ones are:
    multilib: rsync://slackware.uk/people/alien/multilib/
    slackbuilds: rsync://slackware.uk/people/alien/slackbuilds
    sbrepos: rsync://slackware.uk/people/alien/sbrepos
    ktown: rsync://slackware.uk/people/alien-kde/

    Note that by rsync-ing slackware.uk/people/alien with the “H” option will save you gigabytes of space because lots of files in there are hard-linked: “rsync -HrlptD rsync://slackware.uk/people/alien /your/local/storage/”

    And slackware-live should soon appear as: rsync://slackware.uk/people/alien-slacklive

    And take note of this message on the slackware.uk homepage:

    Notice to mirrors:
    If you are running a mirroring service and pulling data from us on a regular basis, please email for access to our un-capped, non-bandwidth limited service – your updates will then be unaffected by bandwidth limitations and user-level capping.

  10. Darren

    Just a note about the ‘Notice to mirrors’ on the site – at the moment I don’t implement any bandwidth capping as the usage has never required it. Rsync itself is limited to 150 connections in total; so unless it’s release day (where I get hammered!) there should be no problems with access.
    I should really edit that MOTD 😀

    If you do find any problems, or need help setting up your rsync script to pull data from the site, feel free to drop me an email (address on the homepage).

    Cheers,
    Darren.

    • Tonus

      Hi Darren,

      Could you tell me the space each repo has on your server ?

      It is asked by a potential mirror maintainer.

      Anyway, thanks for your mirror !

  11. chrisretusn

    I used your taper sight for manually syncing ktown and multilib and only when needed. Just two simple bash scripts with the appropriate rsync options including excludes for the unneeded parts. I was considering adding the live-ISO’s to the mix. I currently do a manual via the browser download.

    Must of been a lot of folks syncing taper to cause those problems. Really appreciate having the alternate locations. Many thanks to Darren for this and to those other fine folks offering up server space. Wish I could do the same.

  12. schu

    Eric, can you remove my last message, I don’t need you to email me anymore. I have the mirror going now:

    http://mirrors.aptalaska.net/slackware/
    rsync://mirrors.aptalaska.net/slackware/

  13. Darren

    Hey,
    Here’s a ‘du’ of all the repos:
    9.7G absolute
    132M csb
    14G gsb
    9.7G microlinux
    1.2G msb
    15G people/alien
    5.4G people/alien-arm
    12G people/alien-kde
    7.9G people/alien-slacklive
    3.6G people/alphageek
    29G people/rworkman
    22G porteus
    107G salix
    63G sbosrcarch
    779M slackbuilds.org
    27G slackintosh
    158G slackware
    23G slackwarearm
    49G slacky
    18G slamd64
    21G slaxbmc
    8.0K slint
    109M sls
    23G splack
    14G truva

    Hope that helps 🙂

  14. Karl

    Thanks Eric and Darren.

    I’ve got the mirrors running now. The URLs are
    rsync://slackware1.gsacrd.ab.ca/mirrors
    rsync://slackware2.gsacrd.ab.ca/mirrors

  15. Drakeo

    are you sure them dates are correct on this mirror.
    http://mirrors.aptalaska.net/slackware/people/alien/multilib/current/slackware64-compat32/x-compat32/
    if so I would wonder.

  16. alienbob

    Drakeo, perhaps something changed on that mirror since you posted your message, but the dates look OK to me.

  17. Darren

    I don’t think it was me that mentioned the date thing.
    But…

    On occasion people may notice ‘old’ files due to the time it takes to sync from Eric’s server (at most i’ve only ever got ~300K/sec), and that I use –delay-updates with rsync since this makes all syncs ‘atomic’

    Well, as atomic as atomic gets with rsync. ‘New’ files are not moved into place until every file is downloaded – this prevents the repos havng some ‘new’ files and ‘old’ ones at the same time. I’d recommend –delay-updates (along with –partial-dir) to anyone running a mirror.

    I can post the full set of options I use if people are interested 🙂

  18. schu

    @Drakeo, yea, dates are correct, I’m -9 GST so it looks like a different day, but it’s not.

    As for mirroring, I’m using:
    rsync -ravlH –delete –no-motd –delay-updates –partial-dir –safe-links

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

© 2024 Alien Pastures

Theme by Anders NorenUp ↑