This site is a work in progress and has not been widely shared. Content may contain errors. Feedback is welcome.
This site is undergoing review. Some annotations were human-generated, some AI-generated — all are being verified.
Back to papers

The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot

Song, Agarwal, Wen

2024SSRN Electronic Journal10 citations
Experimental evidenceInformation SystemsCausal
LLM / Generative AISoftware / codingHuman-AI collaborationCollective intelligence / teams
DOI: 10.2139/ssrn.4856935
Summary

Song, Agarwal, and Wen use proprietary GitHub Copilot usage data combined with public repository data and the Generalized Synthetic Control Method to study how AI pair programmers affect project-level code contributions and coordination time in collaborative open-source software development from 2021-2022.

Main Finding

GitHub Copilot adoption increases project-level code contributions by 5.9% (driven by 2.1% higher individual contributions and 3.4% more developer participation) but also increases coordination time for code integration by 8%, with peripheral developers experiencing smaller productivity gains than core developers due to lower project familiarity.

Primary Datasets

GitHub Innovation Graph

AI-focused

GitHub Archive Dataset (public repository data); GitHub proprietary Copilot usage data

Secondary Datasets

None

Key Methods
Generalized Synthetic Control Method (GSCM) with staggered adoption timing, comparing repositories where Copilot was used (treatment) versus not used (control), validated with propensity score matching and difference-in-differences; analyzes repository-month panel data with two-way fixed effects.
Sample Period
2021-2022
Geographic Coverage
Global (GitHub repositories worldwide)
Sample Size
7,637 repositories (4,491 treatment, 3,146 control) with 139,329 repository-month observations
Level of Analysis
Individual, Firm
Occupation Classification
None
Industry Classification
None
Notes
SSRN Electronic Journal [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I. [Claude classification]: This paper uses a natural experiment design exploiting staggered adoption of GitHub Copilot across repositories. Treatment is defined as repositories where Copilot was both supported by the local IDE and actually used by developers. The analysis uses Generalized Synthetic Control Method to handle multiple treated units with different adoption times. While Copilot is an AI tool, the study itself does not use AI methods for analysis. The paper examines both productivity (code contributions) and coordination costs (merge time), finding a net positive effect on project-level productivity despite increased coordination time. Core developers are defined as those with write access and project control; peripheral developers contribute irregularly without direct codebase access. Key distinction is project familiarity, not skill level. LDA topic modeling used to analyze discussion content diversity in Appendix I.