GitHub Enterprise 2.6 is now deprecated as of April 26, 2017. That means that no patch releases will be made, even for critical security issues, after this date. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
/trending page could incorrectly display a Sign up for free button.GitHub Enterprise 2.6 will be deprecated as of April 26, 2017. That means that no patch releases will be made, even for critical security issues, after this date. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
GitHub Enterprise 2.5 is now deprecated as of March 14, 2017. That means that no patch releases will be made, even for critical security issues, after this release. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
GitHub Enterprise 2.6 will be deprecated as of April 26, 2017. That means that no patch releases will be made, even for critical security issues, after this date. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
/ character.<Destination> element is no longer optional in the SAML response.GitHub Enterprise 2.4 is now deprecated as of February 9, 2017. That means that no patch releases will be made, even for critical security issues, after this release. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
GitHub Enterprise 2.5 will be deprecated as of March 14, 2017. That means that no patch releases will be made, even for critical security issues, after this date. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
A CRITICAL issue was identified that allows an attacker to bypass SAML authentication. The vulnerability is applicable if the attacker has access to a validly signed SAML assertion or response against the configured Verification certificate. When applicable, an attacker can sign in as any user, including administrators.
The affected supported versions are:
Note: This is a different vulnerability than the one addressed in GitHub Enterprise 2.8.6, 2.7.10, 2.6.15, and 2.5.20.
A CRITICAL issue was identified that allows an attacker to execute arbitrary commands on the GitHub Enterprise appliance. The vulnerability is applicable if the attacker has access to configure a repository's Webhooks - owner or admin privileges to a repository.
The affected supported versions are:
We strongly recommend upgrading your GitHub Enterprise appliance to the latest patch release in your series, GitHub Enterprise 2.8.7, 2.7.11, 2.6.16, 2.5.21, or 2.4.23.
Additionally, if SAML authentication is configured in your appliance, all existing SAML user sessions should be destroyed:
Put your GitHub Enterprise environment in Maintenance Mode.
SSH to your primary GitHub Enterprise appliance.
Destroy the existing SAML sessions.
$ echo SAML::Session.destroy_all | ghe-console -y
Upgrade to the latest patch release in your series, GitHub Enterprise 2.8.7, 2.7.11, 2.6.16, 2.5.21, or 2.4.23.
If possible, we also recommend restricting Management Console access to your site administrators.
These vulnerabilities were reported through the GitHub Security Bug Bounty program and we have no evidence that they have been exploited in the wild. To learn more about the Bug Bounty program for GitHub Enterprise, visit https://bounty.github.com/targets/github-enterprise.html and our recent blog post about the inclusion of GitHub Enterprise, Bug Bounty anniversary promotion: bigger bounties in January and February.
Please contact GitHub Enterprise Support if you have any questions.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
A CRITICAL issue was identified that allows an attacker to bypass SAML authentication by creating a fake response. This could allow the attacker to sign in as any user, including administrators.
The affected supported versions are:
If you are using SAML as your authentication method, we strongly recommend upgrading your GitHub Enterprise appliance to the latest patch release in your series, GitHub Enterprise 2.8.6, 2.7.10, 2.6.15, or 2.5.20.
Additionally, all existing user sessions should be destroyed:
Put your GitHub Enterprise environment in Maintenance Mode.
SSH to your primary GitHub Enterprise appliance.
Destroy the existing SAML sessions.
$ echo SAML::Session.destroy_all | ghe-console -y
Upgrade to the latest patch release in your series, GitHub Enterprise 2.8.6, 2.7.10, 2.6.15, or 2.5.20.
This vulnerability was reported through the GitHub Security Bug Bounty program and we have no evidence that it has been exploited in the wild.
Please contact GitHub Enterprise Support if you have any questions.
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
ghe-migrator now scrubs access tokens from the logs.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
babeld service to segment fault under certain circumstances.pre-receive hook would have failed with the error "Something went wrong with the request. Please try again."/setup/api/settings API endpoint failed to apply when applying at the same time as uploading the license for the first time.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
memcached was stopped.ghe-migrator.ghe-update-check utility returned an incorrect message, you must first upgrade to, when it was not necessary.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
git symbolic-ref would hang when resolving references with broken symlinks.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
A CRITICAL issue was identified for all 2.x versions of GitHub Enterprise. The GitHub Enterprise images contain pre-generated SSH host keys that were not regenerated upon installation for all supported platforms:
ssh_host_ed25519_key in GitHub Enterprise section below)This means an attacker with the capability to perform a man-in-the-middle attack on SSH traffic can intercept and modify network traffic to the GitHub Enterprise appliance.
The affected supported versions are:
This vulnerability was found and reported internally and we have no evidence that it has been exploited in the wild.
We strongly recommend upgrading your GitHub Enterprise appliance to the latest patch release in your series, GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, 2.4.17, or 2.3.21. In addition, with backup-utils-2.7.1, ghe-backup and ghe-restore will check for any leaked SSH host keys in the snapshot(s).
Please contact GitHub Enterprise Support if you have questions.
--
If you've upgraded to the latest patch release, GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, 2.4.17, 2.3.21, or greater,
SSH to your primary GitHub Enterprise appliance.
Check for leaked SSH host keys using the ghe-ssh-check-host-keys utility.
$ ghe-ssh-check-host-keys
The utility should output either:
One or more of your SSH host keys were found in the blacklist.
Please reset your host keys using ghe-ssh-roll-host-keys.
--
The SSH host keys were not found in the SSH host key blacklist.
No additional steps are needed/recommended at this time.
If one or more SSH host keys were found in the blacklist, continue to the next step. Otherwise, your GitHub Enterprise environment is not vulnerable.
Put your GitHub Enterprise environment in Maintenance Mode.
Rotate all SSH host keys using the ghe-ssh-roll-host-keys utility.
$ sudo ghe-ssh-roll-host-keys
$ sudo ssh-keygen -t ed25519 -N "" -f /etc/ssh/ssh_host_ed25519_key
The utility should output:
$ SSH host keys have successfully been rolled.
If you've upgraded to GitHub Enterprise 2.7.4, 2.6.9, or greater, and you are using the High Availability Configuration, there are no additional steps to take on your replica appliance.
If you've upgraded to GitHub Enterprise 2.5.14, 2.4.17, 2.3.21, or greater, and you are using the High Availability Configuration,
After completing steps 1-5, stop replication on the replica appliance.
$ ghe-repl-stop
Synchronize the SSH host keys from the primary appliance.
$ ghe-repl-setup
Resume replication on the replica appliance.
$ ghe-repl-start
If you've upgraded to GitHub Enterprise 2.7.4, 2.6.9, 2.5.14 or greater, and you are using Clustering,
After completing steps 1-5, apply the changes to all cluster nodes.
$ ghe-cluster-config-apply
--
If you're unable to upgrade immediately to the latest patch release, GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, 2.4.17, 2.3.21, or greater,
SSH to your primary GitHub Enterprise appliance.
Download the list of leaked SSH host keys and verify its content using any of the provided hashes.
$ curl -O https://enterprise.github.com/security/2016-09-20/ghe-ssh-leaked-host-keys-list.txt
$ sha256sum ghe-ssh-leaked-host-keys-list.txt
3bb29658784a4059a41f1a77cffba9586baab179ba07b795f80e12a9f10c5665 ghe-ssh-leaked-host-keys-list.txt
$ sha1sum ghe-ssh-leaked-host-keys-list.txt
5db799da044da9aae0bcfc523d22e7ce0fe72550 ghe-ssh-leaked-host-keys-list.txt
$ md5sum ghe-ssh-leaked-host-keys-list.txt
de75bcb0bf1d13e15620952c0af8da41 ghe-ssh-leaked-host-keys-list.txt
Print the fingerprint of your GitHub Enterprise appliance's SSH host keys.
Note: The ssh_host_ed25519_key may exist on your GitHub Enterprise appliance but is only used in 2.7.4 or greater.
$ ssh-keygen -lf /etc/ssh/ssh_host_dsa_key.pub
1024 b2:69:82:2f:25:48:bb:fc:62:c7:9a:de:41:42:13:55 /etc/ssh/ssh_host_dsa_key.pub (DSA)
$ ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub
256 c0:cb:fd:07:33:e9:62:14:6b:fb:d5:26:54:f3:c5:0d /etc/ssh/ssh_host_ecdsa_key.pub (ECDSA)
$ ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
256 d6:92:21:4b:04:3b:22:f5:ee:85:0a:63:bf:b3:fe:9b /etc/ssh/ssh_host_ed25519_key.pub (ED25519)
$ ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub
2048 0f:ee:8d:02:2d:e1:76:f3:eb:f5:af:cb:38:9a:1c:33 /etc/ssh/ssh_host_rsa_key.pub (RSA)
Check for leaked SSH host keys by comparing against the downloaded list of leaked SSH host keys.
If one or more SSH host keys were found in the blacklist, continue to the next step. Otherwise, your GitHub Enterprise environment is not vulnerable.
Put your GitHub Enterprise environment in Maintenance Mode.
Remove all SSH host keys.
$ sudo rm -f /etc/ssh/ssh_host_*
Regenerate the SSH host keys.
Note: The ssh_host_ed25519_key may exist on your GitHub Enterprise appliance but is only used and regenerated for in 2.7.4 or greater.
$ sudo ssh-keygen -t ed25519 -N "" -f /etc/ssh/ssh_host_ed25519_key
$ sudo dpkg-reconfigure openssh-server
Apply the changes to the ssh and babeld service.
$ sudo cp /etc/ssh/ssh_host_{rsa,dsa,ecdsa,ed25519}_key{,.pub} /data/user/common/
$ sudo chown babeld:babeld /data/user/common/ssh_host_{rsa,dsa,ecdsa,ed25519}_key{,.pub}
If you're unable to upgrade immediately to GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, 2.4.17, 2.3.21, or greater, and you are using the High Availability Configuration,
$ ghe-repl-stop
$ ghe-repl-setup
$ ghe-repl-start
If you're unable to upgrade immediately to GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, or greater, and you are using Clustering,
$ ghe-cluster-config-apply
--
After rotating the SSH host keys, your GitHub Enterprise environment can exit Maintenance Mode.
Your end-users will receive an error message when attempting to use the Administrative Shell (SSH) or the SSH protocol for Git activity. The rotation does not affect users using the HTTPS protocol for Git activity.
For example, the following is an output from the command-line,
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:seFT9eIOmAZWbfcO9yU1sXiEYIqcrdi0qttbtmNm0Io.
Please contact your system administrator.
Add correct host key in /Users/monalisa/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/monalisa/.ssh/known_hosts:42
ECDSA host key for [github.example.com]:122 has changed and you have requested strict checking.
Host key verification failed.
After updating the known_hosts, end-users will be prompted to accept a new fingerprint.
$ ssh -p 122 admin@github.example.com
The authenticity of host '[github.example.com]:122 ([169.254.1.1]:122)' can't be established.
ECDSA key fingerprint is SHA256:seFT9eIOmAZWbfcO9yU1sXiEYIqcrdi0qttbtmNm0Io.
Are you sure you want to continue connecting (yes/no)?
We strongly recommend publishing your GitHub Enterprise appliance's SSH host key fingerprints in a location that is accessible to all your end-users. For example, for GitHub.com, we publish the SSH fingerprints at https://help.github.com/articles/what-are-github-s-ssh-key-fingerprints/.
If you'd like to to give end-users notice before rotating the SSH host keys, follow the instructions in the Verification and Mitigation if Immediate Upgrade is not Possible skipping step 7 and replacing step 8 with,
Regenerate the SSH host keys.
Note: The ssh_host_ed25519_key may exist on your GitHub Enterprise appliance but is only used and regenerated for in 2.7.4 or greater.
i. Pre-generate new SSH host keys to a temporary directory.
$ ssh-keygen -t dsa -N "" -f /var/tmp/ssh_host_dsa_key
$ ssh-keygen -t rsa -N "" -f /var/tmp/ssh_host_rsa_key
$ ssh-keygen -t ecdsa -N "" -f /var/tmp/ssh_host_ecdsa_key
$ ssh-keygen -t ed25519 -N "" -f /var/tmp/ssh_host_ed25519_key
ii. Print the fingerprint of your GitHub Enterprise appliance's SSH host keys for tentative rotation.
$ ssh-keygen -lf /var/tmp/ssh_host_dsa_key.pub
1024 b2:69:82:2f:25:48:bb:fc:62:c7:9a:de:41:42:13:55 /var/tmp/ssh_host_dsa_key.pub (DSA)
$ ssh-keygen -lf /var/tmp/ssh_host_ecdsa_key.pub
256 c0:cb:fd:07:33:e9:62:14:6b:fb:d5:26:54:f3:c5:0d /var/tmp/ssh_host_ecdsa_key.pub (ECDSA)
$ ssh-keygen -lf /var/tmp/ssh_host_ed25519_key.pub
256 d6:92:21:4b:04:3b:22:f5:ee:85:0a:63:bf:b3:fe:9b /var/tmp/ssh_host_ed25519_key.pub (ED25519)
$ ssh-keygen -lf /var/tmp/ssh_host_rsa_key.pub
248 0f:ee:8d:02:2d:e1:76:f3:eb:f5:af:cb:38:9a:1c:33 /var/tmp/ssh_host_rsa_key.pub (RSA)
iii. Once you are ready to migrate to the new, rotated SSH host keys, move the host keys from the temporary directory and apply the changes to the ssh service.
$ sudo mv /var/tmp/ssh_host_{rsa,dsa,ecdsa,ed25519}_key{,.pub} /etc/ssh
$ sudo service ssh restart
iv. Continue with steps 9 in the Verification and Mitigation if Immediate Upgrade is not Possible section.
ssh_host_ed25519_key in GitHub EnterpriseThe 2.x versions of GitHub Enterprise on all supported platforms:
contained a pre-generated ssh_host_ed25519_key. However, only GitHub Enterprise 2.7.4 or greater use the ssh_host_ed25519_key. This can be verified by checking your GitHub Enterprise appliance's /etc/ssh/sshd_config, which added HostKey /etc/ssh/ssh_host_ed25519_key in 2.7.4 or greater.
The ssh_host_ed25519_key may exist on your GitHub Enterprise appliance but is only used in 2.7.4 or greater.
If you've upgraded your appliance to 2.7.4 or greater on any of the supported platforms including Amazon Web Services, please follow the instructions in the Verification and Mitigation on GitHub Enterprise 2.7.4, 2.6.9, 2.5.14, 2.4.17, 2.3.21, or greater section.
Internal Server Error.ghe-system-info command line utility was not available to run because the utility was missing from the $PATH.ghe-cluster-config-check command line utility terminated early from unsuccessful cURL checks.Not Found when the trailing slash was omitted.ghe-ssl-ca-certificate-install command line utility did not accept a piped certificate as input.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.ssh_host_ed25519_key in GitHub Enterprise for GitHub Enterprise 2.7.4 or greater appliances on the Amazon Web Services platform. (updated 2016-09-22)Thanks!
The GitHub Team
git-lfs pull could cause high MySQL CPU usage.babeld, did not scale the number of workers when memory was added./tmp permissions.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
curl and/or gpg command may have failed using the default hook environment due to missing libraries.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
rbenv, used by many components of GitHub Enterprise, have been tightened.ghe-user-suspend command.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
ghe-migrator did not include issue file attachments, which could cause imports to another server to fail.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
?w=1 to the URL.ghe-import-redis or setting up a cluster, could fail if reading in the data takes longer than 30 seconds to complete.ghe-migrator did not include issue file attachments, which could cause imports to another server to fail.-y argument to ghe-upgrade.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.Thanks!
The GitHub Team
$GITHUB_REPO_PUBLIC variable wasn't available to pre-receive hook scripts when edits were made via the web UI.ghe-migrator failed to import users without an email address, which could cause the whole import to fail.ghe-cluster-support-bundle -t [ticket reference] failed on a GitHub Enterprise Cluster.ghe-storage-extend could fail.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed.ghe-migrator does not include issue file attachments, which may cause imports to another server to fail. (updated 2016-06-09)Thanks!
The GitHub Team
languages queue weren't run. This caused repository language statistics to be inaccurate.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed. (updated 2016-05-24)ghe-migrator does not include issue file attachments, which may cause imports to another server to fail. (updated 2016-06-09)Thanks!
The GitHub Team
Several vulnerabilities in ImageMagick, a package commonly used by web services to process images, have been discovered and disclosed by members of the Mail.ru Security team. One of the vulnerabilities is critical and can lead to remote code execution when processing user submitted images.
Final patches for all the disclosed vulnerabilities within ImageMagick are still pending. This release mitigates the remote code execution vulnerability by implementing the recommended policy to disable the vulnerable ImageMagick coders.
This vulnerability exists in ImageMagick but there is no evidence that it has been exploited on GitHub Enterprise.
We strongly recommend that all GitHub Enterprise customers upgrade their instances as soon as possible.
Mitigation
If you can't immediately upgrade, the issue can be mitigated by implementing the policy changes as follows:
SSH to your GitHub Enterprise appliance.
Edit the /etc/ImageMagick/policy.xml file:
sudo vi /etc/ImageMagick/policy.xml
Disable the vulnerable coders by replacing the <policymap> section with:
<policymap>
<policy domain="coder" rights="none" pattern="EPHEMERAL" />
<policy domain="coder" rights="none" pattern="URL" />
<policy domain="coder" rights="none" pattern="HTTPS" />
<policy domain="coder" rights="none" pattern="MVG" />
<policy domain="coder" rights="none" pattern="MSL" />
</policymap>
There is no need to reboot or restart any services; the changes will take effect immediately.
Please contact GitHub Enterprise Support if you have any questions.
authorized_keys file every time the configuration was saved.find command was missing in the default pre-receive hook environment.languages queue aren't run. This causes repository language statistics to be inaccurate.svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed. (updated 2016-05-24)ghe-migrator does not include issue file attachments, which may cause imports to another server to fail. (updated 2016-06-09)Thanks!
The GitHub Team
With the new features added in GitHub Enterprise 2.6.0, you can:
is:inactive filter.-h or --help.ghe-repl-start will report if high availability replication is still starting following a reboot.ghe-repl-status displays which host is the high availability replica when run on the primary node.ghe-ssl-ca-certificate-install are automatically replicated to the high availability replica.ghe-ssl-ca-certificate-install -l.Upgrading to the 2.6 release series is supported from GitHub Enterprise 2.4.0 and above.
In order to backup and restore GitHub Enterprise 2.6, you will need to upgrade backup-utils to version 2.6.0.
ghe-migrator on the destination appliance.ghe-migrator could fail to import on the destination appliance.ghe-support-bundle displayed harmless messages.GitHub Enterprise 2.1 is now deprecated. That means that no patch releases will be made, even for critical security issues, after this release. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
GitHub Enterprise 2.2 will be deprecated as of August 2016. That means that no patch releases will be made, even for critical security issues, after this date. For better performance, improved security, and new features, upgrade to the newest version of GitHub Enterprise as soon as possible.
Support for Internet Explorer 9 and 10 will be deprecated in a future release. There will be no changes in site functionality, but a warning banner will be displayed to Internet Explorer 9 and 10 users.
GitHub Pages on GitHub Enterprise 2.7 and later will only support kramdown, Jekyll's default Markdown engine. If you are currently using Rdiscount or Redcarpet we've enabled kramdown's GitHub-flavored Markdown support by default, meaning kramdown should have all the features of the two deprecated Markdown engines, so the transition should be as simple as updating the Markdown setting to kramdown in your site's configuration (or removing it entirely).
languages queue aren't run. This causes repository language statistics to be inaccurate. (updated 2015-04-28)find command isn't available in the default pre-receive hook environment. (updated 2015-04-28)svn checkout may timeout while the repository data cache is being built. In most cases, subsequent svn checkout attempts will succeed. (updated 2016-05-24)ghe-migrator does not include issue file attachments, which may cause imports to another server to fail. (updated 2016-06-09)Thanks!
The GitHub Team