Upgrading Netatalk on your DIY Time Capsule and adding better support for Mavericks

Hi!

If you’ve followed my earlier tutorial on how to setup a Linux based Time Capsule, you might want to follow this guide to make sure it’s up to date. As of now, Ubuntu doesn’t have the latest Netatalk available, and you have to install it manually. This includes a bit of manual work, but is fairly simple.

Why would you want to upgrade?

We’re using this setup at my office, and after doing an upgrade to Mavericks, I found it terribly slow! After doing a lot of research, I figured I could try to upgrade the Netatalk version, which was as version 2.2.X. With the upgrade I got at version 3.1.0.

Did it make a difference? Hooooo, yes! It takes a couple of minutes for it to start, but afterwards I copied 95GB in 1,5 hours! It is a bit of a hassle to upgrade, as they’ve removed several of the configuration files and merged them into one. It’s a different structure in the configuration files as well. In addition to having a different structure, they’ve also set a couple of the options as default, meaning less configuration for the user. This will allow for a much simpler scratch install with Netatalk, and a much simpler management of users.

The install

If you are running in a virtual machine, please take a copy of the whole machine. Even though this is fairly straight forward, you never know. Please make sure you have a backup before beginning, or continue on your own risk.

If you’re ready to continue, log in with sudo on your server, and continue:

Begin with stopping the Netatalk service:

service netatalk stop

Then run the following command to install the dependencies. (You will probably have several these from before, but it won’t hurt to be sure):

apt-get install build-essential pkg-config checkinstall git-core avahi-daemon libavahi-client-dev libdb5.1-dev db-util db5.1-util libgcrypt11 libgcrypt11-dev

Then download the latest (3.1.0) Netatalk version:

wget http://prdownloads.sourceforge.net/netatalk/netatalk-3.1.0.tar.bz2?download

Untar it and move to /usr/src/ then change directory to the new location:

tar jxvf netatalk-3.1.0.tar.bz2\?download
mv netatalk-3.1.0 /usr/src/
cd /usr/src/netatalk-3.1.0/

Run the following commands:

./configure --with-init-style=debian --with-zeroconf
make
checkinstall

Now you’re ready to configure the AFP share and user credentials. You should write down, or copy out what you have in the Β bottom of these files:

/etc/netatalk/afpd.conf
/etc/netatalk/AppleVolumes.default

These are now obsolete, and replaced by:

/usr/local/etc/afp.conf

Remember this?:

/mnt/TimeCapsuleData "TimeCapsule" cnidscheme:dbd options:usedots,upriv,tm volsizelimit:100000 allow:timecapsule

A pain to manage, especially with several users. A multiple user one would typically look like this:

/mnt/TimeCapsuleData/user1 "User1" cnidscheme:dbd options:usedots,upriv,tm volsizelimit:100000 allow:user1
/mnt/TimeCapsuleData/user2 "User2" cnidscheme:dbd options:usedots,upriv,tm volsizelimit:100000 allow:user2
/mnt/TimeCapsuleData/user3 "User3" cnidscheme:dbd options:usedots,upriv,tm volsizelimit:100000 allow:user3

And you also had to configure afpd.conf to include:

- -tcp -noddp -uamlist uams_dhx.so,uams_dhx2_passwd.so -nosavepassword

These are now default settings!

The new way of setting up users and shares is by editing /usr/local/etc/afp.conf to look something like this:

;
; Netatalk 3.x configuration file
;

[Global]
mimic model = TimeCapsule6,106
log level = default:warn
log file = /var/log/afpd.log
hosts allow = 10.0.0.0/16

; [Homes]
; basedir regex = /xxxx

[User1]
path = /mnt/TimeCapsuleData/user1
valid users = user1
time machine = yes
vol size limit = 100000

[User2]
path = /mnt/TimeCapsuleData/user2
valid users = user2
time machine = yes
vol size limit = 100000

[User3]
path = /mnt/TimeCapsuleData/user3
valid users = user3
time machine = yes
vol size limit = 100000

Managing this is so much easier! You can also easily add other non-time machine shares as well.

If you only have one common Time Machine login, remove “User2” and “User3” and change “User1” to this:

[TimeMachine]
path = /mnt/TimeCapsuleData/
valid users = timemachine
time machine = yes
vol size limit = 100000

After you’ve done this, Β do a reboot, and you should be good to go!

In theory, this guide should also work for a scratch install, but please take a look at my earlier guide to get some tips on how to mount and format the storage for your DIY Time Capsule.

Advertisements

57 thoughts on “Upgrading Netatalk on your DIY Time Capsule and adding better support for Mavericks

  1. Pingback: Create your very own Linux based Time Capsule for Mac OS X! | A SysAdmins Perspective - Living-On-Line

    • How long did you let it run? It could be slow at first if you are performance a backup of your whole system. Give it 5-10 minutes, just to see if it goes any faster.
      Here it typically takes 1-2 minutes where it goes slowly, then goes up to 50MB/s

      • Ive done the whole system backup before this upgrade, it took me like 6 hours to backup 160GB now its taking for ever to backup 10MB

      • 160GB in 6 hours seems fair.
        Was this before the netatalk upgrade?

        Which OS are you running?
        Cabled/wired network?

        Did you restart the server?

        Check your config file.

        You could try to delete the sparsebundle file from your backup folder (delete your backup) and try again, new full backup.

        I haven’t seen this problem after I upgraded netatalk.

      • yeah it was before the update, wired, gigabit cable. im running Maveriks, server restarted twice.
        What about config file?

        Thanks for the attention given!!

      • now, it did speed up, after 20 minuts of doing nothing, just complete the 11MB backup in like 40 seconds!

  2. Hi. I have problem. When I try to set the backup volume in timemachine settings I get an error after entering the login and password: “OSStatus error 2”. Any ideas? thanks.

      • For some reason I need to log into the host using the “connect to server” option first and then open the shared folder. Then it works. That’s odd but functional. Thanks.

      • I too get this error. I tried doing what Carlos said, but that didn’t solve the problem, because I can’t establish a connection with the server even tha way. Can you please tell me how to fix this issues?

        Thanks!

      • Problem solved! I had set my username wrong. I typed too fast and missed a character…

        Thanks and congrats for this great guide!

      • Thanks!

        Permissions are typically the problem when “strange” things happen with these things.
        Glad you worked it out! πŸ™‚

    • Actually, running “checkinstall” without any arguments does run the command “make install” as default, and as long as you’re logged in as root, everything should be fine.
      Thanks for your comment!

  3. hi! Me again, after a couple of weeks of tests I can tell that after the upgrade the wired speed is boosted and works great but if you are using the wi-fi connection seems to be a little bit slower in realation with wifi speed before the upgrade, I dont know if that makes any sense to you.

    thanks for your time.

    • It will be slower, yes, but you shouldn’t experience a huge difference as long as the WiFi connection is good. I get 150Mbit over WiFi with a 5GHz router, which is fairly good.
      I would do some tests on the router, maybe get one that can handle a lot of load if you experience further problems…

  4. Hi !

    I’ve tried this guide and works like a charm. I have to say that I had to remove the file “/etc/avahi/services/afpd.service” as the new afpd daemon claimed

    Mar 14 00:18:08.894160 afpd[15188] {afp_avahi.c:159} (E:AFPDaemon): Failed to add service: Local name collision

    I removed the old file, made a service netatalk force-reload and everything is working fine now

    Thanks for this great guide!

  5. Hello
    is not working for me on Ubuntu 13.10 server

    os x 9.2 log:
    06/04/14 03:47:07,000 kernel[0]: ASP_TCP CheckReqQueueSize: increasing req queue from 32 to 128 entries. so 0xffffff803b0a7298
    06/04/14 03:47:09,000 kernel[0]: ASP_TCP ReadDSIHeader: setting atten bytes of 0x8000
    06/04/14 03:47:09,000 kernel[0]: ASP_TCP CancelOneRequest: cancelling slot 3 error 89 reqID 5 flags 0x9 afpCmd 0x14 so 0xffffff803b0a7298
    06/04/14 03:47:09,000 kernel[0]: ASP_TCP do_thread_read: no reqInfo found for reqID 256
    06/04/14 03:47:09,397 NetAuthSysAgent[22776]: afp_sendmsg: sendmsg failed 0xffffffff errno = 89
    06/04/14 03:47:09,397 KernelEventAgent[1389]: tid 54485244 received event(s) VQ_SERVEREVENT (2048)
    06/04/14 03:47:09,397 NetAuthSysAgent[22776]: ERROR: AFP_OpenSession – Login failed with 80

    afp log:
    Apr 06 03:46:19.150238 netatalk[13292] {netatalk.c:359} (N:Default): Netatalk AFP server starting
    Apr 06 03:46:19.154633 cnid_metad[13294] {cnid_metad.c:479} (N:AFPDaemon): CNID Server listening on localhost:4700
    Apr 06 03:46:19.400500 afpd[13293] {afp_config.c:125} (N:AFPDaemon): Netatalk AFP/TCP listening on 192.168.101.254:548
    Apr 06 03:46:45.734599 afpd[13300] {afp_dsi.c:106} (N:AFPDaemon): AFP statistics: 0.51 KB read, 0.38 KB written

    afp conf:
    ;
    ; Netatalk 3.x configuration file
    ;

    [Global]
    afp listen = 192.168.101.254
    mimic model = TimeCapsule6,106
    log file = /var/log/afp.log
    hosts allow = 127.0.0.1 192.168.101.0/24

    [polaftp]
    path = /home/poladesign/FTP
    valid users = poladesign

    I don’t know what is wrong?

      • I haven’t seen this error before, but it seems like an authentication failure. Could it be a wrong username/password? Insufficient permissions?

  6. Hi, thanks for the info. Could you please list the steps required client-side? Yesterday I modified the afp.conf adding timemachine = yes and the size limit. I did not use “valid users”,because the backup is of the whole Mac and therefore is located on a common folder. However, I cannot see the volume listed in the TM preference pane. It is true that I didn’t reboot the server because it was being used, but in the past the changes to the afp.conf are detected without reboot/restart of the service. Maybe I am missing something client-side?
    Thanks

  7. I followed your guide, it work great ty.
    But, the mac (on Mavericks) won’t see more than 2.05TB of storage, anyone as a hint on what to do?
    I have 6TB available.

    • I just figured that using the default number of sectors in fdisk must cause my problem, but I tried different ending sector number but it doesn’t work.
      How can I calculate number of sectors?
      Any other solution to increase the size of the time capsule?

  8. Hi
    Thanks for the tutorial πŸ™‚

    I have a little problem with the netatalk service.
    When I power on or reboot the Ubuntu Server I always have to start the netatalk service manually “sudo service netatalk start”

    The Ubuntu Server is running in the background on a virtual machine using VW Ware 10

    Is there a way to start the netatalk service automatically (autostart ?)

    I’m using Ubuntu 14.04 and Netatalk 3.1.2

    Thank you
    Best regards
    Ayman

  9. Starting Netatalk services: /usr/local/sbin/netatalk: error while loading shared libraries: libatalk.so.12: cannot open shared object file: No such file or directory

    • AFAIK this is a bug in Netatalk, did you figure it out or do you still have the same problem? Unfortunately I haven’t tried with newer versions of Ubuntu, but I know several others have had success with the latest versions of both Netatalk and Ubuntu.

  10. Hi, Its me again, just to say tanks again, your guides helped me a lot. Wanna cheer you on keep the page up, it was really healpful!
    It has been almost a year since the last new guide or project!! Keep it up! new projects new fun more people helped! πŸ˜€

    • Hey!
      Thanks! I appreciate it! Good to hear it’s running stable. I did think about renewing my guides to later OS’s, but i think they would pretty much the same.
      Unfortunately I haven’t had the time for new projects lately, but don’t worry, I got some things coming up. Mainly R/C stuff atm. but I got multiple interests, hopefully more of my posts will be interesting for you πŸ™‚
      Keep coming in!

  11. I’ve used your guide with success! I’m doing my first huge backup of about 400Gb with an estimated time of 13 hours. Is it normal more or less?
    Thanks a lot
    Massimo

    • Congrats on finishing your setup!
      Well, the initial backup does take a longer time to start, indexing etc. But after that it’s pretty much just copying.
      How are you connected, wireless or wired? What are you writing to/from?
      Consider your numbers: 400.000MB / 13 hours = 30769MB / 3600 seconds = 8.55MB/s.
      This is a tenth of what a typical hard drive should be able to read or write. Of course any older drive would be about 50MB/s or even lower.
      8.55 * 8 = 68.5Mbit. This is reasonable if you’re on a wireless network, but a cables network is typically at least 100mbit, a modern network 1Gbit.

      If you list the specifics of your setup, it might be easier to see where it slows down. I’m typically at 50MB/s when backuping larger files. Time Machine in general is a bit slower than normal file transfer.

      • Hi Sys
        Sorry but I was very happy so I forgot technical details πŸ˜‰
        I’m connected from my iMac (100mb) to the Ubuntu server machine (1gb), wired. The time machine storage partition in on raid 1.
        On the server are up and running your configuration for time machine and a plexmediaserver and torrentbox, but not operating at the same time with tm.
        Yes I no that the first backup i slowly then the others. Actually I’ve completed about 20 backup set without any problem.
        I’m completing my guide to install the whole server. Could I link your guide?
        Thanks

  12. Hi,

    I followed your guide but on a physical server not virtual. All went ok and time machine sees my share. But when trying to connect it says it can’t and I need to check server name or ip?

    I the did this upgrade but it didn’t solve the problem.

    Any ideas?

    Don

    • Physical vs virtual shouldn’t really matter. So you can see the share, but when trying to connect you can’t access it?
      This means that the network is OK – you can see it there, and probably ping it as well.
      Typically problems related to network services are caused because of insufficient permissions.
      Double check the username that you are using, does it have permission to access your share? Are you at all prompted with the username/password dialogue when choosing the network share (time machine AFP)?

      • Hi,

        Yes I am prompted with the dialogue and username is correct.
        Just checked the permissions and they are correct for the user aswell.

        User root is not allowed and when I try with root it doesn’t even come with error message but just says it’s a wrong user/passw.

        Don

      • This is from my phone, so dont know if ita reply or new comment. If new comment im sorry!

        /etc/local/etc/afp.conf

        ; Netatalk 3.x configuration file
        ;

        [Global]
        ; Global server settings
        mimic model = TimeCapsule6, 106
        log level = default:warn
        log file = /var/log/afpd.log

        ; [Homes]
        ; basedir regex = /xxxx

        [DonBucknall]
        path = /mnt/TimeCapsule/
        valid users = donbucknall
        time machine = yes

      • I think I see what’s wrong. You don’t have permission to write to mnt/TimeCapsule if you followed my guide properly. My guide is actually for multiple user logins. I’ll update the blog post with the correct settings.
        Try this:
        Cd /mnt/TimeCapsule/
        mkdir donbucknall
        chown -R donbucknall:donbucknall donbucknall/

        And change DonBucknall path in afp.conf to “/mnt/TimeCapsule/donbucknall”
        I think I’ll update the blog for multiple users only, it’s the best procedure anyways as backups are separated and only available to the owner.

        When you now click “choose disk” and then the afp share, you’ll be able to choose “DonBucknall” as an alternative.

  13. I did as you told, but still I get the same error. I can see the share but it’s called TimeCapsule. (this is in the Time Machine program when I search for disks)

    Do you think it has to do with my setup? Or could it be that I am using Yosemite?

    Don

    • Yosemite works fine with my setup.
      I’m not sure where it fails. Try:
      sudo service netatalk restart

      Maybe netatalk isn’t set to start automatically. Then you should be able to choose “TimeCapsule” as a share, and then the name you set in afp.conf.

      I’ll do a whole new blog post on this, one that’s independent from these other posts, and with the newest Ubuntu and newest netatalk in the near future.

  14. Pingback: Hello World: Linux VM Time machine for Mac backups

  15. Thanks for these guides but Time Machine is unable to see the server. I don’t think I have users set up properly. I’ve set up two users the “new” way described above in the afp.conf file. I can view the server if I log in as root user in the Mac Finder but can’t login as either of the two users.
    Do I need to add the two users to the server and set up directories for them on the server?
    How do I login to the server from my Macs?

    • The users has to be added and directories with correct permissions has to exist for the users. It’s all in the first guide.

      You can use SSH from terminal on your Mac to connect to the server.

  16. Thank you for this post!
    This is very helpful for me!

    One question: in my organization I have many users, and I want to add quota for each user but i don’t want add config for each user. maybe I can set quota for group for each user ?

    I test netatalk 2.2.0:
    /mnt/data/TM volsizelimit:200000 allow:@my_group options:usedots,upriv,tm dperm:0775 fperm:0660 cnidscheme:dbd
    but limit 200G has been set to all group “my_group” not each user ((

    • I would recommend you to install a newer Netatalk version, where they’ve changed the way you administrate the different users. If you want to separate the backups for each user, you will still have to do a manual config adding users etc, which really doesn’t take too long.
      I’m not sure if there is a way to create a standard quota etc, I haven’t had the need for it.

      I’ve managed 20 users with this system, and it didn’t take me more than maybe 5 minutes to add all users to the config-file, which includes the share, user-space, quota etc. You also need to add the users to the system, and create the folders with correct permissions set.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s