throttle iCloud Photo Library?

Murus
Post Reply
hydrochlorix
Posts: 1
Joined: Mon Jun 01, 2015 5:12 pm

throttle iCloud Photo Library?

Post by hydrochlorix » Mon Jun 01, 2015 5:25 pm

Hi all,

I'm new here and I'm interested in Murus but I'm not sure if it is what I need. We have very poor internet service where I live and I can't get cable here. What happens with our DSL line is that when the upload bandwidth is fully saturated say with uploading 30.000 photos to iCloud Photo Library then the download rate will diminish to basically zero. And then I get yelled at by my wife :oops:
In the olden days of OS X there were tools like throttled and so on that I was comfortable using. I thought I'd give Murus a try with the missing ipfw, but it seems that outbound traffic is for a paid license only. Also, I got to admit that I'm not so comfortable setting it up for my specific use case as I thought I might be what with Murus sporting a GUI and all.
So, long story short, is Murus the tool I need to throttle the upload bandwidth of the photos app and if yes how would I need to set this up?

Thanks a ton!

Thomas

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: throttle iCloud Photo Library?

Post by hany » Mon Jun 01, 2015 7:57 pm

Hello Thomas,

very nice topic :)

Short answer: Murus 1.2 does NOT offer what you need, I'm sorry. BUT Murus 1.3 will offer bandwidth throttling :) We are just about to release a beta. BUT...

Long answer:
what we need to overcome the issue you described (typical for asymmetric connections like home adsl) is ACK Prioritization.
Luckily PF supports ACK Prioritization (typically using ALTQ). Unfortunately, Apple's PF implementation does not include ALTQ. AFAIK there is no way to add support to ALTQ in OS X. So ACK Prioritization is not an option on OS X. And is not an option in Murus. :(
There is another way to overcome the issue you described, it's a much less nice solution, but it works. As you suggested, limiting upload bandwidth (avoiding upload link saturation) is a good solution. This will be possible with Murus 1.3, as it will introduce support for Dummynet.
Dummynet is the OS X built-in traffic shaping tool (coming from the old ipfw and adapted to PF - FreeBSD 8 did it for the first time). Basically you set pipes (and optionally queues) to limit download and upload bandwidth.
WIth Murus you will be able to selectively assign dummynet pipes and/or queue to services and NAT groups for both inbound and outbound connections, in both directions.
We will release the first beta VERY soon. Stay tuned :)

Rickycky
Posts: 1
Joined: Wed Jul 08, 2015 2:46 am

Re: throttle iCloud Photo Library?

Post by Rickycky » Wed Jul 08, 2015 2:51 am

Photos on iCloud is not available for downloading directly from iCloud, right? So I just view iCloud photos on PC first, then I find out that I can use other methods to view and download iCloud photos. All photos on iCloud photo library are downloaded on my PC.
Here is how I do it:
Step 1: Click on “Recover from iCloud Backup File”
Step 2: Download iCloud backup file on the computer
Step 3: Preview and recover iCloud photos

Tusaersa
Posts: 1
Joined: Fri Aug 21, 2015 2:00 am

Re: throttle iCloud Photo Library?

Post by Tusaersa » Fri Aug 21, 2015 2:06 am

Rickycky wrote:Photos on iCloud is not available for downloading directly from iCloud, right? So I just view iCloud photos on PC first, then I find out that I can use other methods to view and download iCloud photos. All photos on iCloud photo library are downloaded on my PC.
Here is how I do it:
Step 1: Click on “Recover from iCloud Backup File”
Step 2: Download iCloud backup file on the computer
Step 3: Preview and recover iCloud photos
Quite useful indeed, I use it to scan and extract data on iCloud, then I found lost of photos that missing for a long time. I have successfully retrieve photos from iCloud now.

fnevers
Posts: 1
Joined: Tue Feb 23, 2016 1:22 pm

Re: throttle iCloud Photo Library?

Post by fnevers » Tue Feb 23, 2016 1:25 pm

Hi,

Did anyone ever figure this out? I have tried on multiple occasions and miserably failed every time. The upload throttling is somehow always set up for all Apple cloud services, not just Photos (e.g. iCloud Drive, iTunes Store, etc.)

Cheers,
Fred

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: throttle iCloud Photo Library?

Post by hany » Wed Feb 24, 2016 4:00 pm

The upload throttling is somehow always set up for all Apple cloud services, not just Photos (e.g. iCloud Drive, iTunes Store, etc.)
Yes, pf packet filter (and its dummynet traffic shaping module) does work at network level, not application level. You throttle connections matching specific ip addresses and/or ports, not applications.

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: throttle iCloud Photo Library?

Post by hany » Thu Apr 07, 2016 2:01 pm

iCloud connections are usually targeting IP addresses belonging to the 17.0.0.0/8 network, which is totally assigned to Apple.
You can limit upload and/or download bandwidth for the whole Apple network to throttle down iCloud connections.
To do so you have to create a dummynet pipe in Murus, then add a custom dummynet rule that limits connections from and/or to address 17.0.0.0/8 and ports 80,443 using that pipe. Please note that this will slow down all connections to the Apple network, including Apple web site and web-based services, not only iCloud.

chicagojoe
Posts: 1
Joined: Tue Jul 12, 2016 9:23 pm

Re: throttle iCloud Photo Library?

Post by chicagojoe » Tue Jul 12, 2016 9:55 pm

Figuring out what processes/applications use what services/networks/ports can really be a chore (I guess that's why Apple tries to make things 'just work' for people... the work has to be done somewhere... either by you on the front end or PC/servers/networks on the back end.)

I too had the same thing recently with Apple Photos. However it now seems they are storing data on AWS and possibly MS Azure (tho I have only seen my iMac hitting Amazon servers.)

Using Little Snitch I could see that nsurlsessiond was making a bunch of connections to a number of icloud.com and s3.amazonaws.com servers (ports 80 & 443) making it the most likely culprit for all things iCloud related (drive, photos, photostreams, family sharing, itunes, app data, etc.).

I'm on an uncapped connection but I didn't want it soaking the connection up or down so I thought I'd try limiting it to 10% of the pipe.
First thing is getting all the addresses that Amazon uses for AWS S3: http://docs.aws.amazon.com/general/late ... anges.html

1) Easiest was to download the file, extract just the IP ranges and pipe that in to another file.
2) Create a new Group called amazon_aws and import the previous file in to the addresses section in Murus. Worked like a charm. Possibly the easiest thing I've ever had to do. *Need to pay for Murus Pro to be able to use the new group*
3) Followed the support video on youtube: https://www.youtube.com/watch?v=1t48YVdVcZQ

Post Reply