Export complete photo list

[Lee Armstrong]

At the moment I am using my own PHP code and cURL to make some requests to the API (99 at a time) to iterate through all photos and store in a MySQL database at my end.

 

Each time I run this I go through all of the photos again and that is getting quite large!

 

Is there a way to say only check for photos between a date/time at all so that I don't do the whole lot?  Can this form part of the REST query?  Or is the anchor on the last request valid for the next request at all?  Bearing in mind the last set will more than likely be <99 photos!

 

Let me know if anything is not clear at all.

[Devin]

While you can't specifically request recent photos from a specific date. You can manually create a query that has the same effect.

 

Record the time stamp of the last photo your downloaded. The next time you run your query, read each photo's time stamp and stop processing the results when you find the time stamp from the last request.

 

[Lee Armstrong]

Thanks Devin,

 

But then I am still reading all the photos right?  I already know the latest timestamp so that isn't an issue but don't get what you mean on the stop processing?

[Devin]

Keep in mind that the Photo/recent endpoint is all photos sorted by their creation date so we can use this fact to stop when we see a photo we have processed before. We can do this using the 'createdDate' property of the photo.

 

Here is some pseudo PHP code:

// Value saved from the last time the script was run
$previousTimestamp = 'PREVIOUS_TIME_STAMP';

$result = NingApi::instance()->photo->fetchNRecent(50);

foreach ($result['entry'] as $photo) {

// Is this photo newer than our last run?
if (compareDates($photo['createdDate'], $previousTimestamp) > 0) {

// Save to your database
savetoDatabase($photo);
} else {
// The rest of the photos were processed the last time we ran this script
break;
}
}

// Used for the next run of this script
$previousTimestamp = $photos[0]['createdDate'];

[Lee Armstrong]

Ok I'm still not getting it.

If I had say 5000 photos that method still means I have to go through each one and only store in my db if not exists. I already do this but I obviously hit the API quite a few times and therefore I cannot poll as often as I like.

Can I not use an anchor point or a >date query on the API???

[Lee Armstrong]

So no response?

 

I my eyes the API at the moment is very heavy and I am using more API calls than I need to!

 

The code at my end I can handle that is no issue but I would like to be able to finish a run on photos, revisit it say 6 hours later and somehow (via an anchor or timestamp) get the recent photos!

 

At the moment more API call's are being made than necessary as I am limited to 100 photos per call.

[Lee Armstrong]

No response still!

This would make a huge difference and cut my API calls by 80-90% by adding this!!!

[Lee Armstrong]

??

[Phil McCluskey]

Hi Lee,

Is there a reason why you can't use the Photo/count function to retrieve the number of photos created since x and then do a call to Photo/recent once you've accumulated somewhere in the neighbourhood of 99?

Thanks,

Phil

[Lee Armstrong]

Thanks for this Phil,

 

So will the API ALWAYS return the newest photos first?

 

So instead of me being able to pass the API a date/anchor photo you are suggesting the following if the above is true.

• Call API for 99 photos
• Scan/import/process them
• When I reach a photo that I have seen before then stop else..
• Repeat until the above is true

[Phil McCluskey]

Hi Lee,

Yes. Once you've done the calls to completely import the photos, then you only need to deal with new photos. The API returns new photos first. So depending on the volume of photos being added to your network, you can do a single call to get the newest 99 photos every hour/day and then determine which ones are not in your database and add only those, or, if the frequency of new photos is highly variable, you can use the count call once every hour or so, and then only make a call to recent once you've accumulated a large enough set (up to 99) to import.

Hope that helps!

[Lee Armstrong]

Thanks Phil,

 

There may be >99 so will using the existing anchor method then return the next 99 again in newest to oldest?

 

This looks like I can work with this.

 

Lee

[Phil McCluskey]

Hi Lee,

Yes, that will let you page back through the sets in subsequent calls.