GitLab API v3: Missing Projects in /projects/all Response Despite Admin Access


3 views

When working with GitLab API v3 (/api/v3/projects/all endpoint), some administrators report missing 6-7 projects from the response despite having full access rights. The missing projects consistently appear to be newer than those included in the response.

GitLab Version: 6.4.3
GitLab Shell: 1.8.0
API Version: v3
Ruby: 2.0.0p353
Rails: 4.0.2

After investigating similar cases, I've identified several potential causes:

  • Project cache not being refreshed after creation
  • Elasticsearch indexing delays (if enabled)
  • Permission propagation delays in older GitLab versions
  • API pagination limits not being handled properly

1. Force Cache Refresh:

# Rails console solution
Project.all.each(&:refresh_import_status)

2. Alternative API Endpoint:

GET /api/v3/projects?simple=true&per_page=100&page=1

3. Direct Database Query (for emergencies):

# SQL query to verify project existence
SELECT id, name, path FROM projects WHERE id NOT IN 
(SELECT project_id FROM project_authorizations WHERE user_id = YOUR_USER_ID);

Here's a Python script to handle pagination properly:

import requests

def get_all_projects(private_token):
    projects = []
    page = 1
    while True:
        url = f"https://gitlab.example.com/api/v3/projects?private_token={private_token}&page={page}&per_page=100"
        response = requests.get(url)
        if not response.json():
            break
        projects.extend(response.json())
        page += 1
    return projects

For GitLab 6.4.3 specifically:

  • The /projects/all endpoint had known caching issues
  • Consider upgrading to at least GitLab 7.0 for better API reliability
  • New projects might take up to 15 minutes to appear in API responses
  1. Check production.log for API request processing
  2. Verify Elasticsearch status if enabled
  3. Test with a newly generated access token
  4. Confirm project visibility levels (internal/private)

When working with GitLab API v3 (specifically endpoint /api/v3/projects/all?private_token=xxx), some administrators report missing 6-7 newer projects from the response, despite having full access via Git CLI and web interface. The issue persists even after token refresh.

GitLab Version: 6.4.3
API Version: v3
Ruby: 2.0.0p353
Rails: 4.0.2

The most common scenarios we've observed:

  • Project visibility settings conflicting with API filters
  • Pagination limits silently truncating results
  • Cache synchronization delays for new projects
  • Legacy API version limitations with newer project types

First, verify project visibility through direct API calls:

curl --header "PRIVATE-TOKEN: <your_token>" \
"https://gitlab.example.com/api/v3/projects/<missing_project_id>"

Check pagination by increasing per_page parameter:

GET /api/v3/projects/all?private_token=xxx&per_page=1000

Option 1: Use alternative endpoints

# List all visible projects (including subgroups)
GET /api/v3/projects?membership=true&per_page=1000

# List all projects (admin-only)
GET /api/v3/projects?per_page=1000&simple=true

Option 2: Implement recursive fetching

def fetch_all_projects(token)
  projects = []
  page = 1
  
  loop do
    response = HTTParty.get(
      "https://gitlab.example.com/api/v3/projects/all",
      query: { private_token: token, page: page, per_page: 100 },
      headers: { 'Accept' => 'application/json' }
    )
    
    break unless response.success?
    projects += JSON.parse(response.body)
    page += 1
    break if response.headers['x-total'].to_i <= projects.count
  end
  
  projects
end

The GitLab 6.4.3 API has known limitations with:

  • Project forks visibility
  • Recently transferred projects
  • Projects with special characters in paths

Consider upgrading to API v4 if possible, as it provides more reliable project enumeration.

  • Confirm admin privileges through /api/v3/user endpoint
  • Check project creation dates vs. API cache timestamps
  • Test with both simple and detailed project representations
  • Verify no IP-based access restrictions exist