Drone fails to sync new GitHub repos via the API

#1

Here’s my intended use case:

  • Create a GitHub repository via the GitHub API
  • Wait 30 seconds (retry in 5 minutes if the next step fails)
  • Enable drone builds: POST /api/repos/github_org/repo_name
  • Change the build settings: PATCH gated:true, trusted:true, private:true

The user that creates the repositories is an owner in the organization
and all of the drone API calls are made with this user’s token.

I always get the following results:

time="2017-09-20T08:31:01Z" level=debug msg="Cannot find repository myorg/myrepo. sql: no rows in result set"
time="2017-09-20T08:31:01Z" level=info ip=172.17.0.1 latency=3.432468ms method=POST path="/api/repos/myorg/myrepo" status=404 time="2017-09-20T08:31:01Z" user-agent=

I tried forcing a sync by calling /api/user/repos?all=true&flush=true before the POST request but that didn’t work. However, if I’m logged in with another account and I force sync through the web UI then it does work. Could be OAuth related?

#2

Sorry, I am currently unable to repeat any issues with synchronization. I just created a new repository and then went into the user interface and clicked the “synchronize” button at which point it was visible in the repositories list.

If you continue to experience issues I recommend taking a look at the synchronization code and sending a patch https://github.com/drone/drone/blob/master/server/sync.go

#3

That workflow works for me too.
Can you synchronize via the API using the bearer token?

#4

Yes, the user interface uses the API. This means that any functions available to the user interface can be invoked directly using your personal account token.

#5

So then the only difference is which user was making the API calls (my own account or the bot account that failed). As I said, the user that failed to synchronize is also an owner in the organization so I don’t see what could’ve gone wrong.

Is there some kind of periodic background sync that I could wait for?
I’ll also try a clean install tomorrow.

#6

Here are my recommendations:

  1. use the chrome developer tools network inspector to see how the API call is being invoked, and make sure you are invoking it correctly (correct path, method, etc)
  2. check the database to see if the repository was added. `select * from repos where repo_full_name=‘foo/bar’
  3. synchronize your personal account as well. It is not enough to synchronize with a bot because drone still needs to synchronize your account and synchronize your individual repository permissions.

Is there some kind of periodic background sync that I could wait for?

there is no background sync. When the user requests the repository list (via the UI or API) it will automatically synchronize if the previous sync was more than 72 hours ago.

#7

The appeal of drone for us is the simple API. So I think when there is a POST request for a missing repository, drone should take care of synchronization automatically and not fail with a 404 response. For now, I solved this with multiple delayed retries - calling the refresh API endpoint each time before the POST request.

Also GitHub seems to have organization web hooks. Can’t we utilize that?
I can’t promise anything but I may be able to allocate some time to contribute work on this.
Organization wide repository settings would be pretty cool.

#8

I recommend setting up a simple utility that accepts organization hooks and then invokes the drone endpoint to synchronize repositories.

For example you could setup a simple captain hook configuration:

echo <<EOF > /some/local/config
{
    "scripts": [
        {
            "command": "curl",
            "args": [
                "-x", 
                "POST", 
                "-H", 
                "Authorization: Bearer {token}", 
                "https://drone.company.com/api/user/repos?all=true&flush=true"
            ]
        }
    ]
}
EOF

and then run the docker image:

docker run -d -v /some/local/config:/config bketelsen/captainhook

The reason this is preferred (at this time) to enabling organization webhooks in drone core is that we would need to assess all providers (gitlab, gitea, gogs, bitbucket, stash, coding) and implement the functionality across the board, if available.

#9

Oh, right, there are many providers. We talked about this on gitter once.
I’ll go with your recommendation then.