overhaul -> overhaul (#21)

* overhaul the repository

add protobuf

change repo url

try to restore github python syntax highlighter

move breakpad into third_party, update packagescript

* AMBuildifying

remove unnecessary files

Move the git patching into ambuild

move lss to a patch

Add windows compilation support

remove breakpad.bat

move postlink libs

* Overhaul CI (#4)

* Dockerbuild (#5)

* make cwd_cmd spew stdout and stderr

* add proper docker build support

* Overhaul ci (#6)

* Setup CI

* fix checkout

* fix yaml syntax

* no fail fast

* setup CI cache

* Fix pip install

* remove pip git

* update actions, ditch node 16

* small syntax cleanups

* more CI changes

* github doc lied

---------

Co-authored-by: Kenzzer <kenzzer@users.noreply.github.com>

* final push for perfect dockerbuilds in every scenario that i have been able to find

* rename cicd->dockerbuild

---------

Co-authored-by: Kenzzer <kenzzer@users.noreply.github.com>

* Add readme, remode duplicate -fPIC

update names of dockerbuild folder in sh files

* cleanup dockerfile (#7)

* Update 0002-Write-FUNC-records-instead-of-PUBLIC-for-ELF-symbols.patch

* Statically link libz, libgcc & libstdc++

* fix submodule path

* Review change + comment patch

---------

Co-authored-by: Kenzzer <kenzzer@users.noreply.github.com>
Co-authored-by: Benoist <14257866+Kenzzer@users.noreply.github.com>
This commit is contained in:
sappho 2024-10-10 21:59:28 -04:00 committed by GitHub
parent 1a72e9e520
commit 1d60f3746a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
37 changed files with 6509 additions and 1110 deletions

102
.github/workflows/build-extension.yml vendored Normal file
View File

@ -0,0 +1,102 @@
name: Build extension
on:
workflow_call:
inputs:
os:
type: string
required: true
cc:
type: string
required: true
cxx:
type: string
upload:
type: boolean
required: false
default: false
upload-artifact-name:
type: string
required: false
default: package
debug:
type: boolean
required: false
default: false
cache-key:
type: string
required: true
cache-dir:
type: string
required: true
jobs:
build:
name: Accelerator Extension ${{ inputs.os }}-${{ inputs.cc }}
runs-on: ${{ inputs.os }}
env:
SOURCEMOD: ${{ github.workspace }}/${{ inputs.cache-dir }}/sourcemod
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: true
- name: Retrieve the cache
id: cache
uses: actions/cache@v4
with:
path: ${{ inputs.cache-dir }}
key: ${{ inputs.cache-key }}
fail-on-cache-miss: true
enableCrossOsArchive: true
- name: Linux dependencies
if: startsWith(runner.os, 'Linux')
run: |
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc-multilib g++-multilib libstdc++6 lib32stdc++6 \
libc6-dev libc6-dev-i386 linux-libc-dev \
linux-libc-dev:i386 lib32z1-dev \
zlib1g-dev:i386 zlib1g-dev ${{ inputs.cc }}
- name: Select clang compiler
if: startsWith(runner.os, 'Linux')
run: |
echo "CC=${{ inputs.cc }}" >> $GITHUB_ENV
echo "CXX=${{ inputs.cxx }}" >> $GITHUB_ENV
${{ inputs.cc }} --version
${{ inputs.cxx }} --version
- uses: actions/setup-python@v5
name: Setup Python 3.10
with:
python-version: '3.10'
- name: Install AMBuild
run: |
python -m pip install --upgrade pip setuptools wheel
pip install ./${{ inputs.cache-dir }}/ambuild
- name: Build (Debug)
if: ${{ inputs.debug }}
run: |
mkdir -p build && cd build
python ../configure.py --enable-debug
ambuild
- name: Build (Release)
if: not ${{ inputs.debug }}
run: |
mkdir -p build && cd build
python ../configure.py --enable-optimize
ambuild
- name: Upload package
if: ${{ inputs.upload }}
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.upload-artifact-name }}
path: build/package

57
.github/workflows/cache.yml vendored Normal file
View File

@ -0,0 +1,57 @@
name: Cache
on:
workflow_call:
outputs:
key:
value: ${{ jobs.cache.outputs.key }}
dir:
value: .cache
jobs:
cache:
name: Create cache
runs-on: 'ubuntu-latest'
outputs:
key: ${{ steps.cache-key.outputs.key }}
env:
SOURCEMOD_REF: 'e0e018c5ebcd11df9bab91b43d40bf148805c684'
AMBUILD_REF: '2d4620da4cdaf89cf25afc3577f920e6138ae7cd'
steps:
- uses: actions/checkout@v4
- name: Create cache key
run: |
echo "${{ env.SOURCEMOD_REF }}" >> cache-key
echo "${{ env.AMBUILD_REF }}" >> cache-key
- name: Output cache key
id: cache-key
run: |
ls -a
echo "key=cache-${{ hashFiles('cache-key') }}" >> $GITHUB_OUTPUT
- name: Retrieve the cache
id: cache
uses: actions/cache@v4
with:
path: .cache
key: ${{ steps.cache-key.outputs.key }}
enableCrossOsArchive: true
- uses: actions/checkout@v4
if: ${{ !steps.cache.outputs.cache-hit }}
name: SourceMod checkout
with:
repository: alliedmodders/sourcemod
ref: ${{ env.SOURCEMOD_REF }}
submodules: true
path: .cache/sourcemod
- uses: actions/checkout@v4
if: ${{ !steps.cache.outputs.cache-hit }}
name: AMBuild checkout
with:
repository: alliedmodders/ambuild
ref: ${{ env.AMBUILD_REF }}
path: .cache/ambuild

56
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,56 @@
name: CI
on:
workflow_dispatch:
pull_request:
branches:
- master
- overhaul
push:
branches:
- master
- overhaul
jobs:
cache:
uses: ./.github/workflows/cache.yml
build:
name: Build
needs: cache
strategy:
fail-fast: false
matrix:
os: [ 'ubuntu-20.04', 'ubuntu-latest', 'windows-2019', 'windows-latest' ]
include:
- os: ubuntu-20.04
cc: clang-8
cxx: clang++-8
upload: false
upload-artifact-name: none
- os: ubuntu-latest
cc: clang
cxx: clang++
upload: true
upload-artifact-name: accelerator_linux
- os: windows-2019
cc: msvc
cxx: msvc
upload: false
upload-artifact-name: none
- os: windows-latest
cc: msvc
cxx: msvc
upload: true
upload-artifact-name: accelerator_windows
uses: ./.github/workflows/build-extension.yml
with:
os: ${{ matrix.os }}
cc: ${{ matrix.cc }}
cxx: ${{ matrix.cxx }}
upload: ${{ matrix.upload }}
upload-artifact-name: ${{ matrix.upload-artifact-name }}
cache-key: ${{ needs.cache.outputs.key }}
cache-dir: ${{ needs.cache.outputs.dir }}

2
.gitignore vendored
View File

@ -1,3 +1,3 @@
/build
/extension/version_auto.h
/breakpad
.vscode

6
.gitmodules vendored Normal file
View File

@ -0,0 +1,6 @@
[submodule "breakpad"]
path = third_party/breakpad
url = https://github.com/google/breakpad.git
[submodule "zlib"]
path = third_party/zlib
url = https://github.com/madler/zlib.git

View File

@ -1,23 +0,0 @@
sudo: false
language: cpp
addons:
apt:
packages:
- g++-multilib
- libc6-dev:i386
compiler:
- clang
before_script:
- cd ..
- git clone --depth=1 --branch=1.8-dev --recursive https://github.com/alliedmodders/sourcemod sourcemod-1.8
- git clone --depth=1 --branch=master --recursive https://github.com/alliedmodders/ambuild
- cd ambuild && python setup.py install --user
- cd $TRAVIS_BUILD_DIR
- git fetch --unshallow || true
- ./breakpad.sh && export PATH=$PATH:$PWD/breakpad/build/src/tools/linux/dump_syms/
script:
- mkdir build && cd build
- python ../configure.py --enable-optimize
- python ./build.py
after_success:
- python ../upload.py

View File

@ -1,242 +1,216 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os
import sys
from ambuild.command import Command
from ambuild.command import ShellCommand
from ambuild.command import SymlinkCommand
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os, sys, shutil
class ExtractDebugInfoCommand(Command):
def __init__(self, binary, outfile):
Command.__init__(self)
self.binary = binary
self.outfile = outfile
def ResolveEnvPath(env, folder=None):
if env in os.environ:
path = os.environ[env]
if os.path.isdir(path):
return path
return None
def run(self, runner, job):
if not self.binary.NeedsRelink(self.outfile):
return
if folder:
head = os.getcwd()
oldhead = None
while head != None and head != oldhead:
path = os.path.join(head, folder)
if os.path.isdir(path):
return path
oldhead = head
head, tail = os.path.split(head)
if AMBuild.target['platform'] == 'linux':
job.AddCommand(ShellCommand('objcopy --only-keep-debug ' + self.outfile + ' ' + self.outfile + '.dbg'))
job.AddCommand(ShellCommand('objcopy --strip-debug ' + self.outfile))
job.AddCommand(ShellCommand('objcopy --add-gnu-debuglink=' + os.path.basename(self.outfile) + '.dbg ' + self.outfile))
elif AMBuild.target['platform'] == 'darwin':
job.AddCommand(ShellCommand('dsymutil ' + self.outfile))
job.AddCommand(ShellCommand('strip -S ' + self.outfile))
return None
class SM:
def __init__(self):
self.compiler = Cpp.Compiler()
def Normalize(path):
return os.path.abspath(os.path.normpath(path))
if AMBuild.mode == 'config':
#Detect compilers
self.compiler.DetectAll(AMBuild)
class AcceleratorConfig(object):
def __init__(self):
self.mms_root = None
self.sm_root = None
self.extension = None
self.libz = None
self.libbreakpad_client = None
self.libbreakpad = None
self.libdisasm = None
self.breakpad_patch = None
self.targets = []
self.target_archs = set()
self.breakpad_config = dict()
self.breakpad_patch = None
#Detect variables
envvars = { 'SOURCEMOD': 'sourcemod-1.8' }
if builder.options.targets:
target_archs = builder.options.targets.split(',')
else:
target_archs = ['x86', 'x86_64']
# Finds if a dict with `key` set to `value` is present on the dict of dicts `dictionary`
def findDictByKey(dictionary, key, value):
for index in dictionary:
elem = dictionary[index]
if elem[key] == value:
return (elem, index)
return None
for arch in target_archs:
try:
cxx = builder.DetectCxx(target_arch = arch)
self.target_archs.add(cxx.target.arch)
except Exception as e:
if builder.options.targets:
raise
print('Skipping target {}: {}'.format(arch, e))
continue
self.targets.append(cxx)
for i in envvars:
if i in os.environ:
path = os.environ[i]
if not os.path.isdir(path):
raise Exception('Path for {0} was not found: {1}'.format(i, path))
else:
head = os.getcwd()
oldhead = None
while head != None and head != oldhead:
path = os.path.join(head, envvars[i])
if os.path.isdir(path):
break
oldhead = head
head, tail = os.path.split(head)
if head == None or head == oldhead:
raise Exception('Could not find a valid path for {0}'.format(i))
AMBuild.cache.CacheVariable(i, path)
if not self.targets:
raise Exception('No suitable C/C++ compiler was found.')
@property
def tag(self):
if builder.options.debug == '1':
return 'Debug'
return 'Release'
def retrieve_sm(self):
if builder.options.sm_path:
self.sm_root = builder.options.sm_path
else:
self.sm_root = ResolveEnvPath('SOURCEMOD', 'sourcemod')
#Set up defines
cxx = self.compiler.cxx
if isinstance(cxx, Cpp.CompatGCC):
if isinstance(cxx, Cpp.GCC):
self.vendor = 'gcc'
elif isinstance(cxx, Cpp.Clang):
self.vendor = 'clang'
self.compiler.AddToListVar('CDEFINES', 'stricmp=strcasecmp')
self.compiler.AddToListVar('CDEFINES', '_stricmp=strcasecmp')
self.compiler.AddToListVar('CDEFINES', '_snprintf=snprintf')
self.compiler.AddToListVar('CDEFINES', '_vsnprintf=vsnprintf')
self.compiler.AddToListVar('CFLAGS', '-pipe')
self.compiler.AddToListVar('CFLAGS', '-fno-strict-aliasing')
if (self.vendor == 'gcc' and cxx.majorVersion >= 4) or self.vendor == 'clang':
self.compiler.AddToListVar('CFLAGS', '-fvisibility=hidden')
self.compiler.AddToListVar('CXXFLAGS', '-fvisibility-inlines-hidden')
self.compiler.AddToListVar('CFLAGS', '-Wall')
self.compiler.AddToListVar('CFLAGS', '-Werror')
self.compiler.AddToListVar('CFLAGS', '-Wno-unused')
self.compiler.AddToListVar('CFLAGS', '-Wno-switch')
self.compiler.AddToListVar('CFLAGS', '-Wno-implicit-exception-spec-mismatch')
self.compiler.AddToListVar('CFLAGS', '-msse')
self.compiler.AddToListVar('CFLAGS', '-g3')
self.compiler.AddToListVar('CFLAGS', '-m32')
self.compiler.AddToListVar('POSTLINKFLAGS', '-m32')
self.compiler.AddToListVar('POSTLINKFLAGS', '-Wl,-z,defs')
self.compiler.AddToListVar('CXXFLAGS', '-std=c++11')
self.compiler.AddToListVar('CXXFLAGS', '-fno-threadsafe-statics')
self.compiler.AddToListVar('CXXFLAGS', '-Wno-non-virtual-dtor')
self.compiler.AddToListVar('CXXFLAGS', '-Wno-overloaded-virtual')
if (self.vendor == 'gcc' and cxx.majorVersion >= 4 and cxx.minorVersion >= 3) or \
(self.vendor == 'clang' and cxx.majorVersion >= 3):
self.compiler.AddToListVar('CXXFLAGS', '-Wno-delete-non-virtual-dtor')
if self.vendor == 'clang' and cxx.majorVersion >= 3:
self.compiler.AddToListVar('CXXFLAGS', '-Wno-implicit-exception-spec-mismatch')
self.compiler.AddToListVar('CDEFINES', 'HAVE_STDINT_H')
self.compiler.AddToListVar('CDEFINES', 'GNUC')
self.compiler.AddToListVar('CDEFINES', '_GLIBCXX_USE_CXX11_ABI=0')
if self.vendor == 'gcc':
self.compiler.AddToListVar('CFLAGS', '-mfpmath=sse')
elif isinstance(cxx, Cpp.MSVC):
self.vendor = 'msvc'
if AMBuild.options.debug == '1':
self.compiler.AddToListVar('CFLAGS', '/MTd')
self.compiler.AddToListVar('POSTLINKFLAGS', '/NODEFAULTLIB:libcmt')
else:
self.compiler.AddToListVar('CFLAGS', '/MT')
self.compiler.AddToListVar('CDEFINES', '_CRT_SECURE_NO_DEPRECATE')
self.compiler.AddToListVar('CDEFINES', '_CRT_SECURE_NO_WARNINGS')
self.compiler.AddToListVar('CDEFINES', '_CRT_NONSTDC_NO_DEPRECATE')
self.compiler.AddToListVar('CXXFLAGS', '/EHsc')
self.compiler.AddToListVar('CFLAGS', '/W3')
self.compiler.AddToListVar('CFLAGS', '/nologo')
self.compiler.AddToListVar('CFLAGS', '/Zi')
self.compiler.AddToListVar('CXXFLAGS', '/TP')
self.compiler.AddToListVar('POSTLINKFLAGS', '/DEBUG')
self.compiler.AddToListVar('POSTLINKFLAGS', '/MACHINE:X86')
self.compiler.AddToListVar('POSTLINKFLAGS', '/SUBSYSTEM:WINDOWS')
self.compiler.AddToListVar('POSTLINKFLAGS', 'kernel32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'user32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'gdi32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'winspool.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'comdlg32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'advapi32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'shell32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'ole32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'oleaut32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'uuid.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'odbc32.lib')
self.compiler.AddToListVar('POSTLINKFLAGS', 'odbccp32.lib')
if not self.sm_root or not os.path.isdir(self.sm_root):
raise Exception('Could not find a source copy of SourceMod')
self.sm_root = Normalize(self.sm_root)
def use_auto_versioning(self):
return not builder.options.disable_auto_versioning
def configure_cxx(self, cxx):
if cxx.like('gcc'):
self.configure_gcc(cxx)
elif cxx.family == 'msvc':
self.configure_msvc(cxx)
#Optimization
if AMBuild.options.opt == '1':
self.compiler.AddToListVar('CDEFINES', 'NDEBUG')
if self.vendor == 'gcc' or self.vendor == 'clang':
self.compiler.AddToListVar('CFLAGS', '-O3')
elif self.vendor == 'msvc':
self.compiler.AddToListVar('CFLAGS', '/Ox')
self.compiler.AddToListVar('POSTLINKFLAGS', '/OPT:ICF')
self.compiler.AddToListVar('POSTLINKFLAGS', '/OPT:REF')
# Optimization
if builder.options.opt == '1':
cxx.defines += ['NDEBUG']
#Debugging
if AMBuild.options.debug == '1':
self.compiler.AddToListVar('CDEFINES', 'DEBUG')
self.compiler.AddToListVar('CDEFINES', '_DEBUG')
if self.vendor == 'msvc':
self.compiler.AddToListVar('CFLAGS', '/Od')
self.compiler.AddToListVar('CFLAGS', '/RTC1')
# Debugging
if builder.options.debug == '1':
cxx.defines += ['DEBUG', '_DEBUG']
#Platform-specifics
if AMBuild.target['platform'] == 'linux':
self.compiler.AddToListVar('CDEFINES', '_LINUX')
self.compiler.AddToListVar('CDEFINES', 'POSIX')
self.compiler.AddToListVar('CDEFINES', 'NO_MALLOC_OVERRIDE')
if self.vendor == 'gcc':
self.compiler.AddToListVar('POSTLINKFLAGS', '-static-libgcc')
if self.vendor == 'clang':
self.compiler.AddToListVar('POSTLINKFLAGS', '-lgcc_eh')
elif AMBuild.target['platform'] == 'darwin':
self.compiler.AddToListVar('CDEFINES', 'OSX')
self.compiler.AddToListVar('CDEFINES', '_OSX')
self.compiler.AddToListVar('POSTLINKFLAGS', '-mmacosx-version-min=10.5')
self.compiler.AddToListVar('POSTLINKFLAGS', ['-arch', 'i386'])
self.compiler.AddToListVar('POSTLINKFLAGS', '-lstdc++')
# Platform-specifics
if cxx.target.platform == 'linux':
self.configure_linux(cxx)
elif cxx.target.platform == 'windows':
self.configure_windows(cxx)
# For OS X dylib versioning
import re
productFile = open(os.path.join(AMBuild.sourceFolder, 'product.version'), 'r')
productContents = productFile.read()
productFile.close()
m = re.match('(\d+)\.(\d+)\.(\d+).*', productContents)
if m == None:
self.version = '1.0.0'
else:
major, minor, release = m.groups()
self.version = '{0}.{1}.{2}'.format(major, minor, release)
AMBuild.cache.CacheVariable('version', self.version)
elif AMBuild.target['platform'] == 'windows':
self.compiler.AddToListVar('CDEFINES', 'WIN32')
self.compiler.AddToListVar('CDEFINES', '_WINDOWS')
if self.use_auto_versioning():
cxx.defines += ['GIT_ACTION_BUILD']
def configure_gcc(self, cxx):
cxx.cflags += [
'-fPIC',
'-pipe',
'-fno-strict-aliasing',
'-fvisibility=hidden',
'-fvisibility-inlines-hidden',
'-Wall',
'-Werror',
'-msse'
]
#Finish up
self.compiler.AddToListVar('CDEFINES', 'SOURCEMOD_BUILD')
self.compiler.AddToListVar('CDEFINES', 'SM_GENERATED_BUILD')
self.compiler.AddToListVar('CINCLUDES',
os.path.join(AMBuild.outputFolder, 'includes'))
self.compiler.ToConfig(AMBuild, 'compiler')
AMBuild.cache.CacheVariable('vendor', self.vendor)
self.targetMap = { }
AMBuild.cache.CacheVariable('targetMap', self.targetMap)
else:
self.compiler.FromConfig(AMBuild, 'compiler')
self.targetMap = AMBuild.cache['targetMap']
cxx.cxxflags += [
'-std=c++17',
'-fno-threadsafe-statics',
'-Wno-non-virtual-dtor',
'-Wno-overloaded-virtual',
'-Wno-implicit-exception-spec-mismatch'
]
if AMBuild.target['platform'] == 'windows':
self.compiler.AddToListVar('RCINCLUDES', os.path.join(AMBuild.sourceFolder, 'extension'))
cxx.postlink += ['-pthread', '-static-libstdc++', '-static-libgcc']
def DefaultCompiler(self):
return self.compiler.Clone()
if builder.options.opt == '1':
cxx.cflags += ['-O3']
return
def JobMatters(self, jobname):
file = sys._getframe().f_code.co_filename
if AMBuild.mode == 'config':
self.targetMap[jobname] = file
return True
if len(AMBuild.args) == 0:
return True
if not jobname in AMBuild.args:
return False
def configure_msvc(self, cxx):
def AutoVersion(self, folder, binary):
if AMBuild.target['platform'] == 'windows':
env = {'RCDEFINES': ['BINARY_NAME="' + binary.binaryFile + '"', 'SM_GENERATED_BUILD']}
binary.AddResourceFile(os.path.join(folder, 'version.rc' ), env)
elif AMBuild.target['platform'] == 'darwin' and isinstance(binary, Cpp.LibraryBuilder):
binary.compiler['POSTLINKFLAGS'].extend(['-compatibility_version', '1.0.0'])
binary.compiler['POSTLINKFLAGS'].extend(['-current_version', AMBuild.cache['version']])
else:
return
cxx.cxxflags += [
'/EHsc',
'/std:c++17'
]
def ExtractDebugInfo(self, job, binary):
src = os.path.join('..', AMBuild.outputFolder, job.workFolder, binary.binaryFile)
job.AddCommand(ExtractDebugInfoCommand(binary, src))
return
def configure_linux(self, cxx):
cxx.defines += ['_LINUX', 'POSIX', '_GLIBCXX_USE_CXX11_ABI=0']
return
def configure_windows(self, cxx):
cxx.defines += ['_WINDOWS']
if cxx.target.arch == 'x86':
cxx.defines += ['WIN32']
elif cxx.target.arch == 'x86_64':
cxx.defines += ['WIN64']
return
def configure(self):
self.retrieve_sm()
sm = SM()
globals = {
'SM': sm
}
for cxx in self.targets:
self.configure_cxx(cxx)
AMBuild.Include(os.path.join('buildbot', 'Versioning'), globals)
def configure_extension(self, compiler, context):
compiler.cxxincludes += [
os.path.join(context.currentSourcePath),
os.path.join(self.sm_root, 'public'),
os.path.join(self.sm_root, 'public', 'extensions'),
os.path.join(self.sm_root, 'public', 'amtl', 'amtl'),
os.path.join(self.sm_root, 'public', 'amtl'),
os.path.join(self.sm_root, 'sourcepawn', 'include')
]
FileList = [
['extension', 'AMBuilder'],
['test', 'AMBuilder'],
['buildbot', 'PackageScript'],
['buildbot', 'BreakpadSymbols']
]
def link_libz(self, compiler, context):
for task in self.libz:
if task.target.arch == compiler.target.arch:
compiler.postlink += [os.path.join(context.buildPath, task.binary.path)]
compiler.linkdeps += [task.binary]
return
raise Exception('No suitable build of libz was found.')
def link_libbreakpad_client(self, compiler, context):
for task in self.libbreakpad_client:
if task.target.arch == compiler.target.arch:
compiler.postlink += [os.path.join(context.buildPath, task.binary.path)]
compiler.linkdeps += [task.binary]
return
raise Exception('No suitable build of libbreakpad_client was found.')
def link_libbreakpad(self, compiler, context):
for task in self.libbreakpad:
if task.target.arch == compiler.target.arch:
compiler.postlink += [os.path.join(context.buildPath, task.binary.path)]
compiler.linkdeps += [task.binary]
return
raise Exception('No suitable build of libbreakpad was found.')
def link_libdisasm(self, compiler, context):
for task in self.libdisasm:
if task.target.arch == compiler.target.arch:
compiler.postlink += [os.path.join(context.buildPath, task.binary.path)]
compiler.linkdeps += [task.binary]
return
raise Exception('No suitable build of libdisasm was found.')
def ConfigureLibrary(self, project, compiler, context):
binary = project.Configure(compiler, project.name, '{0} - {1}'.format(self.tag, compiler.target.arch))
binary.compiler.cxxincludes += [
os.path.join(context.currentSourcePath)
]
return binary
def ConfigureExtension(self, project, compiler, context):
binary = self.ConfigureLibrary(project, compiler, context)
self.configure_extension(binary.compiler, context)
return binary
Accelerator = AcceleratorConfig()
Accelerator.configure()
builder.Build(['third_party/Patch', 'third_party/Configure', 'third_party/AMBuilder'], { 'Accelerator': Accelerator })
BuildScripts = ['extension/AMBuilder', 'buildbot/PackageScript']
builder.Build(BuildScripts, { 'Accelerator': Accelerator })
for parts in FileList:
AMBuild.Include(os.path.join(*parts), globals)

1
README.md Normal file
View File

@ -0,0 +1 @@
# Accelerator

View File

@ -1,24 +0,0 @@
version: '{build}'
os: Visual Studio 2015
environment:
GYP_MSVS_VERSION: '2015'
install:
- cd ..
- git clone --depth=1 --branch=1.8-dev --recursive https://github.com/alliedmodders/sourcemod sourcemod-1.8
- git clone --depth=1 --branch=master https://github.com/alliedmodders/ambuild
- cd ambuild
- C:\python27\python.exe setup.py install
- cd %APPVEYOR_BUILD_FOLDER%
- '"%VS140COMNTOOLS%\vsvars32.bat"'
- breakpad.bat
- set PATH=%PATH%;%CD%\breakpad\src\src\tools\windows\dump_syms\Release
build_script:
- mkdir build
- cd build
- C:\python27\python.exe ..\configure.py --enable-optimize
- C:\python27\python.exe build.py
test: off
on_success:
- C:\python27\python.exe ..\upload.py
#on_finish:
# - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

View File

@ -1,70 +0,0 @@
@echo on
IF EXIST C:\Python27\NUL SET PATH=C:\Python27\;%PATH%
python --version
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
@IF EXIST breakpad\NUL GOTO HASBREAKPAD
mkdir breakpad
:HASBREAKPAD
cd breakpad
@IF EXIST depot_tools\NUL GOTO HASDEPOTTOOLS
git clone --depth=1 --branch=master https://chromium.googlesource.com/chromium/tools/depot_tools.git depot_tools
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
:HASDEPOTTOOLS
@IF EXIST src\NUL GOTO HASSRC
cmd /c depot_tools\fetch --nohooks breakpad
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
GOTO DONESRC
:HASSRC
git -C src fetch
git -C src reset --hard origin/master
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cmd /c depot_tools\gclient sync --nohooks
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
:DONESRC
cd src
git config user.name patches
git config user.email patches@localhost
powershell -Command "& {git am -3 --keep-cr $(ls ..\..\patches\*.patch | %% {$_.FullName})}"
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cd ..
@IF EXIST gyp\NUL GOTO HASGYP
git clone --depth=1 --branch=master https://chromium.googlesource.com/external/gyp.git gyp
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
:HASGYP
powershell -Command "& {(Get-Content src\src\build\common.gypi).replace('''WarnAsError'': ''true'',', '''WarnAsError'': ''false'',') | Set-Content src\src\build\common.gypi}"
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cmd /c gyp\gyp.bat --no-circular-check src\src\client\windows\handler\exception_handler.gyp
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
msbuild src\src\client\windows\handler\exception_handler.sln /m /p:Configuration=Release /p:Platform=Win32
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cmd /c gyp\gyp.bat --no-circular-check src\src\client\windows\crash_generation\crash_generation.gyp
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
msbuild src\src\client\windows\crash_generation\crash_generation.sln /m /p:Configuration=Release /p:Platform=Win32
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cmd /c gyp\gyp.bat --no-circular-check src\src\processor\processor.gyp
IF %errorlevel% neq 0 EXIT /b %errorlevel%
@REM msbuild src\src\processor\processor.sln /m /p:Configuration=Release /p:Platform=Win32
@REM IF %errorlevel% neq 0 EXIT /b %errorlevel%
@REM The solution file is currently including a load of broken projects, so just build exactly what we need.
msbuild src\src\processor\processor.vcxproj /m /p:Configuration=Release /p:Platform=Win32
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
msbuild src\src\third_party\libdisasm\libdisasm.vcxproj /m /p:Configuration=Release /p:Platform=Win32
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cmd /c gyp\gyp.bat --no-circular-check src\src\tools\windows\dump_syms\dump_syms.gyp
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
msbuild src\src\tools\windows\dump_syms\dump_syms.sln /m /p:Configuration=Release /p:Platform=Win32
@IF %errorlevel% neq 0 EXIT /b %errorlevel%
cd ..

View File

@ -1,38 +0,0 @@
#!/bin/sh
set -ex
if [ ! -d "breakpad" ]; then
mkdir breakpad
fi
cd breakpad
if [ ! -d "depot_tools" ]; then
git clone --depth=1 --branch=master https://chromium.googlesource.com/chromium/tools/depot_tools.git depot_tools
fi
if [ ! -d "src" ]; then
PYTHONDONTWRITEBYTECODE=1 python2.7 ./depot_tools/fetch.py --nohooks breakpad
else
git -C src fetch
git -C src reset --hard origin/master
PYTHONDONTWRITEBYTECODE=1 python2.7 ./depot_tools/gclient.py sync --nohooks
fi
cd src
git config user.name patches
git config user.email patches@localhost
git am -3 --keep-cr ../../patches/*.patch
cd ..
if [ ! -d "build" ]; then
mkdir build
fi
cd build
../src/configure --enable-m32 CXXFLAGS="-g -O2 -D_GLIBCXX_USE_CXX11_ABI=0"
make src/tools/linux/dump_syms/dump_syms
make src/client/linux/libbreakpad_client.a
make src/libbreakpad.a src/third_party/libdisasm/libdisasm.a

View File

@ -1,127 +1,53 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
# vim: set ts=8 sts=2 sw=2 tw=99 et ft=python:
import os
import shutil
import ambuild.osutil as osutil
from ambuild.command import Command
job = AMBuild.AddJob('package')
# This is where the files will be output to
# package is the default
builder.SetBuildFolder('package')
class DestroyPath(Command):
def __init__(self, folder):
Command.__init__(self)
self.folder = folder
def destroy(self, path):
entries = os.listdir(path)
for entry in entries:
newpath = os.path.join(path, entry)
if os.path.isdir(newpath):
self.destroy(newpath)
os.rmdir(newpath)
elif os.path.isfile(newpath):
os.remove(newpath)
def run(self, runner, job):
runner.PrintOut('rm -rf {0}/*'.format(self.folder))
self.destroy(self.folder)
class CreateFolders(Command):
def __init__(self, folders):
Command.__init__(self)
self.folders = folders
def run(self, runner, job):
for folder in self.folders:
path = os.path.join(*folder)
runner.PrintOut('mkdir {0}'.format(path))
os.makedirs(path)
#Shallow folder copy
class CopyFolder(Command):
def __init__(self, fromList, toList, excludes = []):
Command.__init__(self)
self.fromPath = os.path.join(AMBuild.sourceFolder, *fromList)
self.toPath = os.path.join(*toList)
self.excludes = excludes
def run(self, runner, job):
entries = os.listdir(self.fromPath)
for entry in entries:
if entry in self.excludes:
continue
path = os.path.join(self.fromPath, entry)
if not os.path.isfile(path):
continue
runner.PrintOut('copy {0} to {1}'.format(path, self.toPath))
shutil.copy(path, self.toPath)
#Single file copy
class CopyFile(Command):
def __init__(self, fromFile, toPath):
Command.__init__(self)
self.fromFile = fromFile
self.toPath = toPath
def run(self, runner, job):
runner.PrintOut('copy {0} to {1}'.format(self.fromFile, self.toPath))
shutil.copy(self.fromFile, self.toPath)
folders = [
['addons', 'sourcemod', 'configs'],
['addons', 'sourcemod', 'gamedata'],
['addons', 'sourcemod', 'extensions'],
# Add any folders you need to this list
folder_list = [
'addons/sourcemod/extensions',
'addons/sourcemod/gamedata',
'addons/sourcemod/configs'
]
#Setup
job.AddCommand(DestroyPath(os.path.join(AMBuild.outputFolder, 'package')))
job.AddCommand(CreateFolders(folders))
# Are we build a x86_64 extension ?
for task in Accelerator.extension:
if task.target.arch == 'x86_64':
folder_list += ['addons/sourcemod/extensions/x64']
break
job.AddCommand(CopyFile(os.path.join(AMBuild.sourceFolder, 'accelerator.games.txt'), os.path.join('addons', 'sourcemod', 'gamedata')))
job.AddCommand(CopyFile(os.path.join(AMBuild.sourceFolder, 'accelerator.autoload'), os.path.join('addons', 'sourcemod', 'extensions')))
# Create the distribution folder hierarchy.
folder_map = {}
for folder in folder_list:
norm_folder = os.path.normpath(folder)
folder_map[folder] = builder.AddFolder(norm_folder)
bincopies = []
# Do all straight-up file copies from the source tree.
def CopyFiles(src, dest, files):
if not dest:
dest = src
dest_entry = folder_map[dest]
for source_file in files:
source_path = os.path.join(builder.sourcePath, src, source_file)
builder.AddCopy(source_path, dest_entry)
def CopyFile(src, dest):
dest_entry = folder_map[dest]
source_path = os.path.join(builder.sourcePath, src)
builder.AddCopy(source_path, dest_entry)
def CopyDirContent(src, dest):
dest_entry = folder_map[dest]
for item in os.scandir(os.path.join(builder.sourcePath, src)):
if item.is_file():
builder.AddCopy(item.path, dest_entry)
def AddNormalLibrary(name, dest):
dest = os.path.join('addons', 'sourcemod', dest)
bincopies.append(CopyFile(os.path.join('..', name, name + osutil.SharedLibSuffix()), dest))
# Each platform's version of dump_syms needs the path in a different format.
if AMBuild.target['platform'] == 'linux':
debug_info.append(name + '/' + name + '.so')
elif AMBuild.target['platform'] == 'darwin':
debug_info.append(name + '/' + name + '.dylib.dSYM')
elif AMBuild.target['platform'] == 'windows':
debug_info.append(name + '\\' + name + '.pdb')
def AddExecutable(name, dest):
dest = os.path.join('addons', 'sourcemod', dest)
bincopies.append(CopyFile(os.path.join('..', name, name + osutil.ExecutableSuffix()), dest))
# Each platform's version of dump_syms needs the path in a different format.
if AMBuild.target['platform'] == 'linux':
debug_info.append(name + '/' + name)
elif AMBuild.target['platform'] == 'darwin':
debug_info.append(name + '/' + name + '.dSYM')
elif AMBuild.target['platform'] == 'windows':
debug_info.append(name + '\\' + name + '.pdb')
def AddHL2Library(name, dest):
for i in SM.sdkInfo:
sdk = SM.sdkInfo[i]
if AMBuild.target['platform'] not in sdk['platform']:
continue
AddNormalLibrary(name + '.ext.' + sdk['ext'], dest)
debug_info = []
AddNormalLibrary('accelerator.ext', 'extensions')
AddExecutable('test-crash-dump-generation', 'configs')
job.AddCommandGroup(bincopies)
pdblog = open(os.path.join(AMBuild.outputFolder, 'pdblog.txt'), 'wt')
for pdb in debug_info:
pdblog.write(pdb + '\n')
pdblog.close()
# Copy binaries.
for task in Accelerator.extension:
if task.target.arch == 'x86_64':
builder.AddCopy(task.binary, folder_map['addons/sourcemod/extensions/x64'])
else:
builder.AddCopy(task.binary, folder_map['addons/sourcemod/extensions'])
CopyDirContent('gamedata', 'addons/sourcemod/gamedata')
CopyFile('extension/accelerator.autoload', 'addons/sourcemod/extensions')

View File

@ -1,10 +1,15 @@
# vim: set ts=2 sw=2 tw=99 noet:
import sys
import ambuild.runner as runner
from ambuild2 import run
run = runner.Runner()
run.options.add_option('--enable-debug', action='store_const', const='1', dest='debug',
help='Enable debugging symbols')
run.options.add_option('--enable-optimize', action='store_const', const='1', dest='opt',
help='Enable optimization')
run.Configure(sys.path[0])
parser = run.BuildParser(sourcePath = sys.path[0], api='2.2')
parser.options.add_argument('--sm-path', type=str, dest='sm_path', default=None,
help='Path to SourceMod')
parser.options.add_argument('--enable-debug', action='store_const', const='1', dest='debug',
help='Enable debugging symbols')
parser.options.add_argument('--enable-optimize', action='store_const', const='1', dest='opt',
help='Enable optimization')
parser.options.add_argument('--disable-auto-versioning', action='store_false', dest='disable_auto_versioning',
default=True, help='Disables the auto versioning script')
parser.options.add_argument('--targets', type=str, dest='targets', default=None,
help="Override the target architecture (use commas to separate multiple targets).")
parser.Configure()

1
dockerbuild/.gitkeep Normal file
View File

@ -0,0 +1 @@

View File

@ -0,0 +1,95 @@
#!/bin/bash
set -euxo pipefail
bootstrapEnv()
{
# for squashing git whining when people run with docker instead of rootless podman
git config --global --add safe.directory /
git config --global --add safe.directory "*"
# for preventing any submodule tomfuckery
git submodule update --init --recursive -f
# whack build dir it's literally faster and easier to do this than to cache it
rm -rfv ./build
# clean up patch cruft in breakpad dir so we start from a clean slate no matter what
pushd third_party/breakpad
git reset --hard
git clean -x -f -d
git clean -X -f -d
popd
}
bootstrapPkgs()
{
# we really need to slim this shit down lol
dpkg --add-architecture i386 && \
apt-get update -y && \
apt-get install -y \
--no-install-recommends \
git \
clang \
make \
python3-httplib2 python3-pip \
lib32stdc++-10-dev lib32z1-dev libc6-dev-i386 linux-libc-dev:i386 \
libzstd-dev libzstd-dev:i386 zlib1g-dev zlib1g-dev:i386
# force clang to be our compiler no matter what, hopefully
update-alternatives --install /usr/bin/cc cc /usr/bin/clang 100
update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++ 100
# this is just for logging
cc --version || true
c++ --version || true
}
smBranch="master"
amTempLocation="_am_temp"
succCloneLocation="/accelerator/${amTempLocation}/successful_clone"
bootstrapAM()
{
# need to install ambuild if we already cloned, otherwise checkout-deps will do it 4 us
if test -f "${succCloneLocation}"; then
pip install /accelerator/"${amTempLocation}"/ambuild
return 255;
fi
rm -rf /accelerator/"${amTempLocation}"/ || true
mkdir -p ${amTempLocation} || exit 1
pushd ${amTempLocation} || exit 1
git clone -b ${smBranch} --recursive --depth 1 https://github.com/alliedmodders/sourcemod sourcemod || exit 1
git clone --recursive --depth 1 https://github.com/alliedmodders/ambuild ambuild || exit 1
pip install ./ambuild
# make a blank file so that we don't reclone everything if we don't need to
true > "${succCloneLocation}" || exit 1
popd
}
buildIt()
{
if test ! -d build; then
mkdir -p build
fi
pushd build
CC=clang CXX=clang++ python3 ../configure.py --sm-path=/accelerator/${amTempLocation}/sourcemod/
ambuild
popd
}
###############################
cd /accelerator
bootstrapPkgs
bootstrapEnv
bootstrapAM || true
buildIt

View File

@ -0,0 +1,16 @@
#!/bin/bash
set -euxo pipefail
pwd
ls -la
mkdir /accelerator
cp -av "$(pwd)"/. /accelerator
cd /accelerator
pwd
ls -la
bash ./dockerbuild/_accelerator_docker_build_internal.sh

View File

@ -0,0 +1,49 @@
#!/bin/bash
set -euxo pipefail
dockerimage="debian:11-slim"
# we do this so that we can be agnostic about where we're invoked from
# meaning you can exec this script anywhere and it should work the same
thisiswhereiam=${BASH_SOURCE[0]}
# this should be /whatever/directory/structure/Open-Fortress-Source
script_folder=$( cd -- "$( dirname -- "${thisiswhereiam}" )" &> /dev/null && pwd )
# this should be /whatever/directory/structure/[accelerator_root]/cicd
build_dir="dockerbuild"
pushd "${script_folder}" &> /dev/null || exit 99
# this is relative to our source dir/build
internalscript="_accelerator_docker_build_internal.sh"
# this should always be our accelerator root dir
pushd ../ &> /dev/null
dev_srcdir=$(pwd)
container_rootdir="accelerator"
# add -it flags automatically if in null tty
itflag=""
if [ -t 0 ] ; then
itflag="-it"
else
itflag=""
fi
podman run ${itflag} \
-v "${dev_srcdir}":/"${container_rootdir}" \
-w /${container_rootdir} \
${dockerimage} \
bash ./${build_dir}/${internalscript} "$@"
ecodereal=$?
echo "real exit code ${ecodereal}"
popd &> /dev/null || exit
popd &> /dev/null || exit
exit ${ecodereal}

View File

@ -1,96 +1,58 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os
import ambuild.osutil as osutil
from ambuild.command import SymlinkCommand
from ambuild.command import ShellCommand
from ambuild.command import DirectCommand
import os, sys
def BuildEverything():
if AMBuild.target['platform'] not in ['linux', 'windows']:
return
builder.SetBuildFolder('/')
compiler = SM.DefaultCompiler()
project = builder.LibraryProject('accelerator.ext')
project.sources = [
'extension.cpp',
'MemoryDownloader.cpp',
os.path.join(Accelerator.sm_root, 'public', 'smsdk_ext.cpp')
]
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.sourceFolder, 'extension'))
def AddSourceFilesFromDir(path, files):
list = []
for file in files:
list.append(os.path.join(path, file))
return list
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.cache['SOURCEMOD'], 'public'))
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.cache['SOURCEMOD'], 'public', 'extensions'))
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.cache['SOURCEMOD'], 'sourcepawn', 'include'))
for cxx in Accelerator.targets:
binary = Accelerator.ConfigureExtension(project, cxx, builder)
compiler = binary.compiler
# Wait for breakpad to be patched
compiler.sourcedeps += Accelerator.breakpad_patch
# We depend on breakpad on config
compiler.sourcedeps += Accelerator.breakpad_config[compiler.target.arch]
if AMBuild.target['platform'] in ['linux']:
compiler['POSTLINKFLAGS'].append('-lm')
compiler['POSTLINKFLAGS'].append('-lstdc++')
compiler['POSTLINKFLAGS'].append('-pthread')
compiler.defines += ['HAVE_CONFIG_H']
compiler.cxxincludes += [
os.path.join(builder.sourcePath, 'third_party', 'breakpad', 'src'),
os.path.join(builder.buildPath, 'third_party', 'config', compiler.target.arch),
]
compiler['CDEFINES'].append('HAVE_CONFIG_H')
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.sourceFolder, 'breakpad', 'build', 'src'))
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src'))
name = 'accelerator.ext'
extension = AMBuild.AddJob(name)
binary = Cpp.LibraryBuilder(name, AMBuild, extension, compiler)
binary.AddSourceFiles('extension', [
'extension.cpp',
'MemoryDownloader.cpp',
])
binary.AddSourceFiles(AMBuild.cache['SOURCEMOD'], ['public/smsdk_ext.cpp'])
if AMBuild.target['platform'] in ['linux']:
binary.AddSourceFiles(os.path.join('breakpad', 'src', 'src', 'common'), [
'dwarf_cfi_to_module.cc',
'dwarf_cu_to_module.cc',
'dwarf_line_to_module.cc',
'dwarf_range_list_handler.cc',
'language.cc',
'module.cc',
'path_helper.cc',
'stabs_reader.cc',
'stabs_to_module.cc',
'dwarf/bytereader.cc',
'dwarf/dwarf2diehandler.cc',
'dwarf/dwarf2reader.cc',
'dwarf/elf_reader.cc',
'linux/crc32.cc',
'linux/dump_symbols.cc',
'linux/elf_symbols_to_module.cc',
if compiler.target.platform in ['linux']:
binary.sources += AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, '..', 'third_party', 'breakpad', 'src', 'common'), [
'dwarf_cfi_to_module.cc',
'dwarf_cu_to_module.cc',
'dwarf_line_to_module.cc',
'dwarf_range_list_handler.cc',
'language.cc',
'module.cc',
'path_helper.cc',
'stabs_reader.cc',
'stabs_to_module.cc',
'dwarf/bytereader.cc',
'dwarf/dwarf2diehandler.cc',
'dwarf/dwarf2reader.cc',
'dwarf/elf_reader.cc',
'linux/crc32.cc',
'linux/dump_symbols.cc',
'linux/elf_symbols_to_module.cc',
'linux/breakpad_getcontext.S'
])
if AMBuild.target['platform'] in ['linux']:
libs = [
('libbreakpad_client.a', os.path.join('breakpad', 'build', 'src', 'client', 'linux', 'libbreakpad_client.a')),
('libbreakpad.a', os.path.join('breakpad', 'build', 'src', 'libbreakpad.a')),
('libdisasm.a', os.path.join('breakpad', 'build', 'src', 'third_party', 'libdisasm', 'libdisasm.a')),
]
Accelerator.link_libbreakpad_client(compiler, builder)
Accelerator.link_libbreakpad(compiler, builder)
Accelerator.link_libdisasm(compiler, builder)
for lib, target in libs:
link = os.path.join(AMBuild.outputFolder, extension.workFolder, lib)
target = os.path.join(AMBuild.sourceFolder, target)
try:
os.lstat(link)
except:
extension.AddCommand(SymlinkCommand(link, target))
binary.AddObjectFiles([lib])
elif AMBuild.target['platform'] in ['windows']:
libs = [
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'handler', 'Release', 'lib', 'common.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'handler', 'Release', 'lib', 'exception_handler.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'crash_generation', 'Release', 'lib', 'crash_generation_client.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'third_party', 'libdisasm', 'Release', 'lib', 'libdisasm.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'processor', 'Release', 'lib', 'processor.lib'),
]
for path in libs:
if os.path.isfile(path):
binary.RelinkIfNewer(path)
binary['POSTLINKFLAGS'].extend([path])
SM.AutoVersion('extension', binary)
SM.ExtractDebugInfo(extension, binary)
binary.SendToJob()
BuildEverything()
Accelerator.extension = builder.Add(project)

View File

@ -115,10 +115,10 @@ SMEXT_LINK(&g_accelerator);
IWebternet *webternet;
IGameConfig *gameconfig;
typedef void (*GetSpew_t)(char *buffer, unsigned int length);
typedef void (*GetSpew_t)(char *buffer, size_t length);
GetSpew_t GetSpew;
#if defined _WINDOWS
typedef void(__fastcall *GetSpewFastcall_t)(char *buffer, unsigned int length);
typedef void(__fastcall *GetSpewFastcall_t)(char *buffer, size_t length);
GetSpewFastcall_t GetSpewFastcall;
#endif
@ -504,10 +504,13 @@ class UploadThread: public IThread
#if defined _LINUX
bool UploadSymbolFile(const google_breakpad::CodeModule *module, const char *presubmitToken) {
if (log) fprintf(log, "UploadSymbolFile\n");
if (log) fflush(log);
auto debugFile = module->debug_file();
std::string vdsoOutputPath = "";
if (debugFile == "linux-gate.so") {
if (false && debugFile == "linux-gate.so") {
FILE *auxvFile = fopen("/proc/self/auxv", "rb");
if (auxvFile) {
char vdsoOutputPathBuffer[512];
@ -523,7 +526,11 @@ class UploadThread: public IThread
if (auxvEntryId == 0) break;
if (auxvEntryId != 33) continue; // AT_SYSINFO_EHDR
#ifdef PLATFORM_X64
Elf64_Ehdr *vdsoHdr = (Elf64_Ehdr *)auxvEntryValue;
#else
Elf32_Ehdr *vdsoHdr = (Elf32_Ehdr *)auxvEntryValue;
#endif
auto vdsoSize = vdsoHdr->e_shoff + (vdsoHdr->e_shentsize * vdsoHdr->e_shnum);
void *vdsoBuffer = malloc(vdsoSize);
memcpy(vdsoBuffer, vdsoHdr, vdsoSize);
@ -548,6 +555,7 @@ class UploadThread: public IThread
}
if (log) fprintf(log, "Submitting symbols for %s\n", debugFile.c_str());
if (log) fflush(log);
auto debugFileDir = google_breakpad::DirName(debugFile);
std::vector<string> debug_dirs{
@ -557,18 +565,19 @@ class UploadThread: public IThread
};
std::ostringstream outputStream;
google_breakpad::DumpOptions options(ALL_SYMBOL_DATA, true);
google_breakpad::DumpOptions options(ALL_SYMBOL_DATA, true, true);
{
StderrInhibitor stdrrInhibitor;
if (!WriteSymbolFile(debugFile, debug_dirs, options, outputStream)) {
if (!WriteSymbolFile(debugFile, debugFile, "Linux", debug_dirs, options, outputStream)) {
outputStream.str("");
outputStream.clear();
// Try again without debug dirs.
if (!WriteSymbolFile(debugFile, {}, options, outputStream)) {
if (!WriteSymbolFile(debugFile, debugFile, "Linux", {}, options, outputStream)) {
if (log) fprintf(log, "Failed to process symbol file\n");
if (log) fflush(log);
return false;
}
}
@ -607,6 +616,7 @@ class UploadThread: public IThread
if (!symbolUploaded) {
if (log) fprintf(log, "Symbol upload failed: %s (%d)\n", xfer->LastErrorMessage(), xfer->LastErrorCode());
if (log) fflush(log);
return false;
}
@ -619,7 +629,7 @@ class UploadThread: public IThread
}
if (log) fprintf(log, "Symbol upload complete: %s\n", response);
delete[] response;
if (log) fflush(log);
return true;
}
#endif
@ -636,6 +646,7 @@ class UploadThread: public IThread
}
if (log) fprintf(log, "Submitting binary for %s\n", codeFile.c_str());
if (log) fflush(log);
IWebForm *form = webternet->CreateForm();
@ -665,6 +676,7 @@ class UploadThread: public IThread
if (!binaryUploaded) {
if (log) fprintf(log, "Binary upload failed: %s (%d)\n", xfer->LastErrorMessage(), xfer->LastErrorCode());
if (log) fflush(log);
return false;
}
@ -676,6 +688,7 @@ class UploadThread: public IThread
response[--responseSize] = '\0';
}
if (log) fprintf(log, "Binary upload complete: %s\n", response);
if (log) fflush(log);
delete[] response;
return true;
@ -971,12 +984,14 @@ class UploadThread: public IThread
if (!submitSymbols && !submitBinary) {
continue;
}
if (log) fprintf(log, "Getting module at index %d\n", moduleIndex);
if (log) fflush(log);
auto module = processState.modules()->GetModuleAtIndex(moduleIndex);
auto moduleType = ClassifyModule(module);
if (log) fprintf(log, "Classified module %s as %s\n", module->code_file().c_str(), ModuleTypeCode[moduleType]);
if (log) fflush(log);
switch (moduleType) {
case kMTUnknown:
continue;
@ -1009,6 +1024,8 @@ class UploadThread: public IThread
#endif
}
}
if (log) fprintf(log, "PresubmitCrashDump complete\n");
if (log) fflush(log);
delete[] response;
return presubmitResponse;

View File

@ -40,7 +40,7 @@
#define SMEXT_CONF_NAME "Accelerator"
#define SMEXT_CONF_DESCRIPTION "SRCDS Crash Handler"
#define SMEXT_CONF_VERSION SM_FULL_VERSION
#define SMEXT_CONF_AUTHOR "Asher \"asherkin\" Baker"
#define SMEXT_CONF_AUTHOR "asherkin, kenzzer"
#define SMEXT_CONF_URL "https://crash.limetech.org/"
#define SMEXT_CONF_LOGTAG "CRASH"
#define SMEXT_CONF_LICENSE "GPL"

View File

@ -11,7 +11,7 @@
{
"windows" "2"
"linux" "2"
"mac" "2"
"linux64" "2"
}
}
@ -25,6 +25,7 @@
{
"library" "engine"
"linux" "@_Z7GetSpewPcj"
"linux64" "@_Z7GetSpewPcm"
}
}
}

View File

@ -1,7 +1,7 @@
From 1f35257db226d6d0189cff0832a97cccc639c91a Mon Sep 17 00:00:00 2001
From: Asher Baker <asherkin@limetech.io>
Date: Sun, 13 Jan 2019 12:34:45 +0000
Subject: [PATCH 1/4] Ignore invalid modules rather than bailing on the entire
Subject: [PATCH 1/5] Ignore invalid modules rather than bailing on the entire
module list
---
@ -9,10 +9,10 @@ Subject: [PATCH 1/4] Ignore invalid modules rather than bailing on the entire
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/src/processor/minidump.cc b/src/processor/minidump.cc
index 78faf77a..c4fda7b7 100644
index 83e5a868..a7d564f2 100644
--- a/src/processor/minidump.cc
+++ b/src/processor/minidump.cc
@@ -2774,7 +2774,7 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
@@ -3195,7 +3195,7 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
BPLOG(ERROR) << "MinidumpModuleList could not read required module "
"auxiliary data for module " <<
module_index << "/" << module_count;
@ -21,7 +21,7 @@ index 78faf77a..c4fda7b7 100644
}
// It is safe to use module->code_file() after successfully calling
@@ -2786,7 +2786,14 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
@@ -3207,7 +3207,14 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
BPLOG(ERROR) << "MinidumpModuleList found bad base address for module "
<< module_index << "/" << module_count << ", "
<< module.code_file();
@ -37,7 +37,7 @@ index 78faf77a..c4fda7b7 100644
}
// Some minidumps have additional modules in the list that are duplicates.
@@ -2813,7 +2820,7 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
@@ -3234,7 +3241,7 @@ bool MinidumpModuleList::Read(uint32_t expected_size) {
<< module_index << "/" << module_count << ", "
<< module.code_file() << ", " << HexString(base_address)
<< "+" << HexString(module_size);
@ -46,6 +46,3 @@ index 78faf77a..c4fda7b7 100644
}
// If failed due to apparent range overlap the cause may be the client
--
2.21.0

View File

@ -1,22 +1,22 @@
From 8aaf6e84a6704eb538f68a3e6fb6c3a8c93f1d8d Mon Sep 17 00:00:00 2001
From: Asher Baker <asherkin@limetech.io>
Date: Sun, 13 Jan 2019 12:35:05 +0000
Subject: [PATCH 2/4] Write FUNC records instead of PUBLIC for ELF symbols with
Subject: [PATCH 2/5] Write FUNC records instead of PUBLIC for ELF symbols with
sizes
---
---
src/common/linux/elf_symbols_to_module.cc | 17 +++++++++++++----
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/src/common/linux/elf_symbols_to_module.cc b/src/common/linux/elf_symbols_to_module.cc
index 562875e1..e11a5265 100644
index 70d50f89..f21460bf 100644
--- a/src/common/linux/elf_symbols_to_module.cc
+++ b/src/common/linux/elf_symbols_to_module.cc
@@ -156,19 +156,28 @@ bool ELFSymbolsToModule(const uint8_t *symtab_section,
@@ -163,19 +163,28 @@ bool ELFSymbolsToModule(const uint8_t* symtab_section,
while(!iterator->at_end) {
if (ELF32_ST_TYPE(iterator->info) == STT_FUNC &&
iterator->shndx != SHN_UNDEF) {
- Module::Extern *ext = new Module::Extern(iterator->value);
- auto ext = std::make_unique<Module::Extern>(iterator->value);
- ext->name = SymbolString(iterator->name_offset, strings);
+ string name = SymbolString(iterator->name_offset, strings);
#if !defined(__ANDROID__) // Android NDK doesn't provide abi::__cxa_demangle.
@ -33,18 +33,15 @@ index 562875e1..e11a5265 100644
#endif
+#if 1
+ if (iterator->size) {
+ Module::Function *fun = new Module::Function(name, iterator->value);
+ Module::Function *fun = new Module::Function(module->AddStringToPool(name), iterator->value);
+ fun->ranges.push_back(Module::Range(iterator->value, iterator->size));
+ module->AddFunction(fun);
+ }
+#else
+ Module::Extern *ext = new Module::Extern(iterator->value);
+ ext->name = name;
module->AddExtern(ext);
module->AddExtern(std::move(ext));
+#endif
}
++iterator;
}
--
2.21.0

View File

@ -1,28 +0,0 @@
From 21dfe5a22da02ed8bcd9fa1cca7c18c8c5310172 Mon Sep 17 00:00:00 2001
From: patches <patches@localhost>
Date: Sun, 28 Apr 2019 18:45:57 +0000
Subject: [PATCH 3/4] Avoid using _ZNSsC1ERKSsjRKSaIcE GLIBCXX symbol due to
version compat issues
---
src/common/dwarf/elf_reader.cc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/common/dwarf/elf_reader.cc b/src/common/dwarf/elf_reader.cc
index 4135a51a..bfaaeefe 100644
--- a/src/common/dwarf/elf_reader.cc
+++ b/src/common/dwarf/elf_reader.cc
@@ -1202,8 +1202,8 @@ const char *ElfReader::GetSectionInfoByName(const string &section_name,
bool ElfReader::SectionNamesMatch(const string &name, const string &sh_name) {
if ((name.find(".debug_", 0) == 0) && (sh_name.find(".zdebug_", 0) == 0)) {
- const string name_suffix(name, strlen(".debug_"));
- const string sh_name_suffix(sh_name, strlen(".zdebug_"));
+ const string name_suffix(&name.c_str()[strlen(".debug_")]);
+ const string sh_name_suffix(&sh_name.c_str()[strlen(".zdebug_")]);
return name_suffix == sh_name_suffix;
}
return name == sh_name;
--
2.21.0

View File

@ -0,0 +1,25 @@
From 2f217812634d5e6c56f0cf9e7a4c9b7fb390e954 Mon Sep 17 00:00:00 2001
From: patches <patches@localhost>
Date: Sat, 20 Jul 2019 15:52:37 +0100
Subject: [PATCH 3/5] Support compilation on VS 2015
---
src/common/windows/pe_util.cc | 1 +
src/common/windows/pe_util.h | 2 +-
src/tools/windows/dump_syms/dump_syms.cc | 1 +
3 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/src/common/windows/pe_util.h b/src/common/windows/pe_util.h
index 6c6b364f..80aba5e5 100644
--- a/src/common/windows/pe_util.h
+++ b/src/common/windows/pe_util.h
@@ -59,7 +59,7 @@ wstring GenerateDebugIdentifier(DWORD age, DWORD signature);
// Converts |machine| enum value to the corresponding string used by Breakpad.
// The enum is IMAGE_FILE_MACHINE_*, contained in winnt.h.
-constexpr const wchar_t* FileHeaderMachineToCpuString(WORD machine) {
+static inline const wchar_t* FileHeaderMachineToCpuString(WORD machine) {
switch (machine) {
case IMAGE_FILE_MACHINE_I386: {
return L"x86";

View File

@ -0,0 +1,38 @@
Subject: [PATCH 4/5] Fixes path to a.out.h file
diff --git a/configure b/configure
index 3442e796..3e7c321c 100755
--- a/configure
+++ b/configure
@@ -7532,11 +7532,16 @@ then :
printf "%s\n" "#define STDC_HEADERS 1" >>confdefs.h
fi
-ac_fn_c_check_header_compile "$LINENO" "a.out.h" "ac_cv_header_a_out_h" "$ac_includes_default"
+ac_fn_c_check_header_compile "$LINENO" "linux/a.out.h" "ac_cv_header_a_out_h" "$ac_includes_default"
if test "x$ac_cv_header_a_out_h" = xyes
then :
printf "%s\n" "#define HAVE_A_OUT_H 1" >>confdefs.h
-
+else
+ ac_fn_c_check_header_compile "$LINENO" "a.out.h" "ac_cv_header_a_out_h" "$ac_includes_default"
+ if test "x$ac_cv_header_a_out_h" = xyes
+ then :
+ printf "%s\n" "#define HAVE_A_OUT_H_EX 1" >>confdefs.h
+ fi
fi
ac_fn_c_check_header_compile "$LINENO" "sys/mman.h" "ac_cv_header_sys_mman_h" "$ac_includes_default"
if test "x$ac_cv_header_sys_mman_h" = xyes
diff --git a/src/common/stabs_reader.h b/src/common/stabs_reader.h
index 655683f1..1e6a76b8 100644
--- a/src/common/stabs_reader.h
+++ b/src/common/stabs_reader.h
@@ -52,6 +52,8 @@
#ifdef HAVE_MACH_O_NLIST_H
#include <mach-o/nlist.h>
#elif defined(HAVE_A_OUT_H)
+#include <linux/a.out.h>
+#elif defined(HAVE_A_OUT_H_EX)
#include <a.out.h>
#endif

View File

@ -1,51 +0,0 @@
From 2f217812634d5e6c56f0cf9e7a4c9b7fb390e954 Mon Sep 17 00:00:00 2001
From: patches <patches@localhost>
Date: Sat, 20 Jul 2019 15:52:37 +0100
Subject: [PATCH 4/4] Support compilation on VS 2015
---
src/common/windows/pe_util.cc | 1 +
src/common/windows/pe_util.h | 2 +-
src/tools/windows/dump_syms/dump_syms.cc | 1 +
3 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/src/common/windows/pe_util.cc b/src/common/windows/pe_util.cc
index 6fa63fa3..03dcd142 100644
--- a/src/common/windows/pe_util.cc
+++ b/src/common/windows/pe_util.cc
@@ -35,6 +35,7 @@
#include <ImageHlp.h>
#include <functional>
+#include <memory>
#include "common/windows/string_utils-inl.h"
#include "common/windows/guid_string.h"
diff --git a/src/common/windows/pe_util.h b/src/common/windows/pe_util.h
index 634ba293..3eefa761 100644
--- a/src/common/windows/pe_util.h
+++ b/src/common/windows/pe_util.h
@@ -60,7 +60,7 @@ wstring GenerateDebugIdentifier(DWORD age, DWORD signature);
// Converts |machine| enum value to the corresponding string used by Breakpad.
// The enum is IMAGE_FILE_MACHINE_*, contained in winnt.h.
-constexpr const wchar_t* FileHeaderMachineToCpuString(WORD machine) {
+static inline const wchar_t* FileHeaderMachineToCpuString(WORD machine) {
switch (machine) {
case IMAGE_FILE_MACHINE_I386: {
return L"x86";
diff --git a/src/tools/windows/dump_syms/dump_syms.cc b/src/tools/windows/dump_syms/dump_syms.cc
index 5b7d1777..1f22cfc2 100644
--- a/src/tools/windows/dump_syms/dump_syms.cc
+++ b/src/tools/windows/dump_syms/dump_syms.cc
@@ -33,6 +33,7 @@
#include <stdio.h>
#include <wchar.h>
+#include <memory>
#include <string>
#include "common/windows/pdb_source_line_writer.h"
--
2.21.0

5342
patches/0005-Add-LSS.patch Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,89 +0,0 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os
import ambuild.osutil as osutil
from ambuild.command import SymlinkCommand
from ambuild.command import ShellCommand
from ambuild.command import DirectCommand
def BuildEverything():
if AMBuild.target['platform'] not in ['linux', 'windows']:
return
compiler = SM.DefaultCompiler()
if AMBuild.target['platform'] in ['linux']:
compiler['POSTLINKFLAGS'].append('-lm')
compiler['POSTLINKFLAGS'].append('-lstdc++')
compiler['POSTLINKFLAGS'].append('-pthread')
compiler['CDEFINES'].append('HAVE_CONFIG_H')
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.sourceFolder, 'breakpad', 'build', 'src'))
elif compiler.cc.name == 'msvc':
compiler['POSTLINKFLAGS'].remove('/SUBSYSTEM:WINDOWS')
compiler['POSTLINKFLAGS'].append('/SUBSYSTEM:CONSOLE')
compiler['CXXINCLUDES'].append(os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src'))
name = 'test-crash-dump-generation'
extension = AMBuild.AddJob(name)
binary = Cpp.ExecutableBuilder(name, AMBuild, extension, compiler)
binary.AddSourceFiles('test', [
'test.cpp',
]);
if AMBuild.target['platform'] in ['linux']:
binary.AddSourceFiles(os.path.join('breakpad', 'src', 'src', 'common'), [
'dwarf_cfi_to_module.cc',
'dwarf_cu_to_module.cc',
'dwarf_line_to_module.cc',
'dwarf_range_list_handler.cc',
'language.cc',
'module.cc',
'path_helper.cc',
'stabs_reader.cc',
'stabs_to_module.cc',
'dwarf/bytereader.cc',
'dwarf/dwarf2diehandler.cc',
'dwarf/dwarf2reader.cc',
'dwarf/elf_reader.cc',
'linux/crc32.cc',
'linux/dump_symbols.cc',
'linux/elf_symbols_to_module.cc',
])
if AMBuild.target['platform'] in ['linux']:
libs = [
('libbreakpad_client.a', os.path.join('breakpad', 'build', 'src', 'client', 'linux', 'libbreakpad_client.a')),
('libbreakpad.a', os.path.join('breakpad', 'build', 'src', 'libbreakpad.a')),
('libdisasm.a', os.path.join('breakpad', 'build', 'src', 'third_party', 'libdisasm', 'libdisasm.a')),
]
for lib, target in libs:
link = os.path.join(AMBuild.outputFolder, extension.workFolder, lib)
target = os.path.join(AMBuild.sourceFolder, target)
try:
os.lstat(link)
except:
extension.AddCommand(SymlinkCommand(link, target))
binary.AddObjectFiles([lib])
elif AMBuild.target['platform'] in ['windows']:
libs = [
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'handler', 'Release', 'lib', 'common.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'handler', 'Release', 'lib', 'exception_handler.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'client', 'windows', 'crash_generation', 'Release', 'lib', 'crash_generation_client.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'third_party', 'libdisasm', 'Release', 'lib', 'libdisasm.lib'),
os.path.join(AMBuild.sourceFolder, 'breakpad', 'src', 'src', 'processor', 'Release', 'lib', 'processor.lib'),
]
for path in libs:
if os.path.isfile(path):
binary.RelinkIfNewer(path)
binary['POSTLINKFLAGS'].extend([path])
SM.ExtractDebugInfo(extension, binary)
binary.SendToJob()
BuildEverything()

View File

@ -1,326 +0,0 @@
#if defined _LINUX
#include "client/linux/handler/exception_handler.h"
#include "common/linux/linux_libc_support.h"
#include "third_party/lss/linux_syscall_support.h"
#include "common/linux/dump_symbols.h"
#include "common/path_helper.h"
#include <signal.h>
#include <dirent.h>
#include <unistd.h>
#include <paths.h>
#define my_jmp_buf sigjmp_buf
#define my_setjmp(x) sigsetjmp(x, 0)
#define my_longjmp siglongjmp
using google_breakpad::MinidumpDescriptor;
using google_breakpad::WriteSymbolFile;
class StderrInhibitor
{
FILE *saved_stderr = nullptr;
public:
StderrInhibitor() {
saved_stderr = fdopen(dup(fileno(stderr)), "w");
if (freopen(_PATH_DEVNULL, "w", stderr)) {
// If it fails, not a lot we can (or should) do.
// Add this brace section to silence gcc warnings.
}
}
~StderrInhibitor() {
fflush(stderr);
dup2(fileno(saved_stderr), fileno(stderr));
fclose(saved_stderr);
}
};
#elif defined _WINDOWS
#define _STDINT // ~.~
#include "client/windows/handler/exception_handler.h"
#define my_jmp_buf jmp_buf
#define my_setjmp(x) setjmp(x)
#define my_longjmp longjmp
#else
#error Bad platform.
#endif
#include <google_breakpad/processor/minidump.h>
#include <google_breakpad/processor/minidump_processor.h>
#include <google_breakpad/processor/process_state.h>
#include <google_breakpad/processor/call_stack.h>
#include <google_breakpad/processor/stack_frame.h>
#include <processor/pathname_stripper.h>
#include <sstream>
#include <streambuf>
#include <setjmp.h>
using google_breakpad::ExceptionHandler;
using google_breakpad::Minidump;
using google_breakpad::MinidumpProcessor;
using google_breakpad::ProcessState;
using google_breakpad::ProcessResult;
using google_breakpad::CallStack;
using google_breakpad::StackFrame;
using google_breakpad::PathnameStripper;
using google_breakpad::CodeModule;
class ClogInhibitor
{
std::streambuf *saved_clog = nullptr;
public:
ClogInhibitor() {
saved_clog = std::clog.rdbuf();
std::clog.rdbuf(nullptr);
}
~ClogInhibitor() {
std::clog.rdbuf(saved_clog);
}
};
my_jmp_buf envbuf;
char path[1024];
#if defined _LINUX
static bool dumpCallback(const MinidumpDescriptor &descriptor, void *context, bool succeeded)
{
if (succeeded) {
sys_write(STDOUT_FILENO, "Wrote minidump to: ", 19);
} else {
sys_write(STDOUT_FILENO, "Failed to write minidump to: ", 29);
}
sys_write(STDOUT_FILENO, descriptor.path(), my_strlen(descriptor.path()));
sys_write(STDOUT_FILENO, "\n", 1);
my_strlcpy(path, descriptor.path(), sizeof(path));
my_longjmp(envbuf, 1);
return succeeded;
}
#elif defined _WINDOWS
static bool dumpCallback(const wchar_t *dump_path, const wchar_t *minidump_id, void *context, EXCEPTION_POINTERS *exinfo, MDRawAssertionInfo *assertion, bool succeeded)
{
if (succeeded) {
printf("Wrote minidump to: %ls\\%ls.dmp\n", dump_path, minidump_id);
} else {
printf("Failed to write minidump to: %ls\\%ls.dmp\n", dump_path, minidump_id);
}
// TODO: setjmp/longjmp doesn't play nicely with SEH on Windows, so we never get back.
// But the exception handler is called and writes the dump, so the user can just invoke us again.
// snprintf(path, sizeof(path), "%ls\\%ls.dmp", dump_path, minidump_id);
// my_longjmp(envbuf, 1);
return succeeded;
}
#endif
int main(int argc, char *argv[])
{
bool shouldDumpSymbols = false;
if (argc > 1 && strcmp(argv[1], "-d") == 0) {
shouldDumpSymbols = true;
}
bool generateCrash = false;
if (argc <= (shouldDumpSymbols ? 2 : 1)) {
generateCrash = true;
if (my_setjmp(envbuf) == 0) {
#if defined _LINUX
MinidumpDescriptor descriptor(".");
ExceptionHandler *handler = new ExceptionHandler(descriptor, NULL, dumpCallback, NULL, true, -1);
#elif defined _WINDOWS
ExceptionHandler *handler = new ExceptionHandler(L".", NULL, dumpCallback, NULL, ExceptionHandler::HANDLER_ALL);
#endif
volatile int *ptr = (volatile int *)(0xdeadbeef);
*ptr = 0;
delete handler;
return 0;
}
printf("Returned from signal handler, path: %s\n", path);
argc = (shouldDumpSymbols ? 3 : 2);
}
for (int i = (shouldDumpSymbols ? 2 : 1); i < argc; ++i) {
if (!generateCrash) {
strncpy(path, argv[i], sizeof(path));
}
ProcessState processState;
ProcessResult processResult;
MinidumpProcessor minidumpProcessor(nullptr, nullptr);
{
ClogInhibitor clogInhibitor;
processResult = minidumpProcessor.Process(path, &processState);
}
if (processResult != google_breakpad::PROCESS_OK) {
continue;
}
std::string os_short = "";
std::string cpu_arch = "";
if (processState.system_info()) {
os_short = processState.system_info()->os_short;
if (os_short.empty()) {
os_short = processState.system_info()->os;
}
cpu_arch = processState.system_info()->cpu;
}
int requestingThread = processState.requesting_thread();
if (requestingThread == -1) {
requestingThread = 0;
}
const CallStack *stack = processState.threads()->at(requestingThread);
if (!stack) {
continue;
}
int frameCount = stack->frames()->size();
if (frameCount > 1024) {
frameCount = 1024;
}
std::ostringstream summaryStream;
summaryStream << 2 << "|" << processState.time_date_stamp() << "|" << os_short << "|" << cpu_arch << "|" << processState.crashed() << "|" << processState.crash_reason() << "|" << std::hex << processState.crash_address() << std::dec << "|" << requestingThread;
std::map<const CodeModule *, unsigned int> moduleMap;
unsigned int moduleCount = processState.modules() ? processState.modules()->module_count() : 0;
for (unsigned int moduleIndex = 0; moduleIndex < moduleCount; ++moduleIndex) {
auto module = processState.modules()->GetModuleAtIndex(moduleIndex);
moduleMap[module] = moduleIndex;
auto debugFile = PathnameStripper::File(module->debug_file());
auto debugIdentifier = module->debug_identifier();
summaryStream << "|M|" << debugFile << "|" << debugIdentifier;
}
for (int frameIndex = 0; frameIndex < frameCount; ++frameIndex) {
const StackFrame *frame = stack->frames()->at(frameIndex);
int moduleIndex = -1;
auto moduleOffset = frame->ReturnAddress();
if (frame->module) {
moduleIndex = moduleMap[frame->module];
moduleOffset -= frame->module->base_address();
}
summaryStream << "|F|" << moduleIndex << "|" << std::hex << moduleOffset << std::dec;
}
auto summaryLine = summaryStream.str();
printf("%s\n", summaryLine.c_str());
#if defined _LINUX
for (unsigned int moduleIndex = 0; shouldDumpSymbols && moduleIndex < moduleCount; ++moduleIndex) {
auto module = processState.modules()->GetModuleAtIndex(moduleIndex);
auto debugFile = module->debug_file();
std::string vdsoOutputPath = "";
if (debugFile == "linux-gate.so") {
FILE *auxvFile = fopen("/proc/self/auxv", "rb");
if (auxvFile) {
auto workingDir = getcwd(nullptr, 0);
vdsoOutputPath = workingDir + std::string("/linux-gate.so");
free(workingDir);
while (!feof(auxvFile)) {
int auxvEntryId = 0;
fread(&auxvEntryId, sizeof(auxvEntryId), 1, auxvFile);
long auxvEntryValue = 0;
fread(&auxvEntryValue, sizeof(auxvEntryValue), 1, auxvFile);
if (auxvEntryId == 0) break;
if (auxvEntryId != 33) continue; // AT_SYSINFO_EHDR
Elf32_Ehdr *vdsoHdr = (Elf32_Ehdr *)auxvEntryValue;
auto vdsoSize = vdsoHdr->e_shoff + (vdsoHdr->e_shentsize * vdsoHdr->e_shnum);
void *vdsoBuffer = malloc(vdsoSize);
memcpy(vdsoBuffer, vdsoHdr, vdsoSize);
FILE *vdsoFile = fopen(vdsoOutputPath.c_str(), "wb");
if (vdsoFile) {
fwrite(vdsoBuffer, 1, vdsoSize, vdsoFile);
fclose(vdsoFile);
debugFile = vdsoOutputPath;
}
free(vdsoBuffer);
break;
}
fclose(auxvFile);
}
}
if (debugFile[0] != '/') {
continue;
}
// printf("%s\n", debugFile.c_str());
auto debugFileDir = google_breakpad::DirName(debugFile);
std::vector<string> debug_dirs{
debugFileDir,
debugFileDir + "/.debug",
"/usr/lib/debug" + debugFileDir,
};
std::ostringstream outputStream;
google_breakpad::DumpOptions options(ALL_SYMBOL_DATA, true);
{
StderrInhibitor stdrrInhibitor;
if (!WriteSymbolFile(debugFile, debug_dirs, options, outputStream)) {
outputStream.str("");
outputStream.clear();
// Try again without debug dirs.
if (!WriteSymbolFile(debugFile, {}, options, outputStream)) {
// TODO: Something.
continue;
}
}
}
// WriteSymbolFileHeaderOnly would do this for us, but this is just for testing.
auto output = outputStream.str();
output = output.substr(0, output.find("\n"));
printf("%s\n", output.c_str());
if (debugFile == vdsoOutputPath) {
unlink(vdsoOutputPath.c_str());
}
}
#endif
}
if (generateCrash) {
unlink(path);
}
return 0;
}

173
third_party/AMBuilder vendored Normal file
View File

@ -0,0 +1,173 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os, sys
def AddSourceFilesFromDir(path, files):
list = []
for file in files:
list.append(os.path.join(path, file))
return list
builder.SetBuildFolder('./third_party')
libz = builder.StaticLibraryProject('libz')
libz.sources = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'zlib'),[
'adler32.c',
'compress.c',
'crc32.c',
'deflate.c',
'gzclose.c',
'gzlib.c',
'gzread.c',
'gzwrite.c',
'infback.c',
'inffast.c',
'inflate.c',
'inftrees.c',
'trees.c',
'uncompr.c',
'zutil.c'
])
for cxx in Accelerator.targets:
if cxx.target.platform == 'linux':
binary = Accelerator.ConfigureLibrary(libz, cxx, builder)
binary.compiler.cflags += ['-Wno-implicit-function-declaration']
Accelerator.libz = builder.Add(libz)
libbreakpad_client = builder.StaticLibraryProject('libbreakpad_client')
libbreakpad_client.sources = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src'),[
'common/convert_UTF.cc',
'common/string_conversion.cc',
])
libbreakpad_client_linux = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src'),[
'client/minidump_file_writer.cc',
'client/linux/crash_generation/crash_generation_client.cc',
'client/linux/crash_generation/crash_generation_server.cc',
'client/linux/dump_writer_common/thread_info.cc',
'client/linux/dump_writer_common/ucontext_reader.cc',
'client/linux/handler/exception_handler.cc',
'client/linux/handler/minidump_descriptor.cc',
'client/linux/log/log.cc',
'client/linux/microdump_writer/microdump_writer.cc',
'client/linux/minidump_writer/linux_core_dumper.cc',
'client/linux/minidump_writer/linux_dumper.cc',
'client/linux/minidump_writer/linux_ptrace_dumper.cc',
'client/linux/minidump_writer/minidump_writer.cc',
'client/linux/minidump_writer/pe_file.cc',
'common/linux/elf_core_dump.cc',
'common/linux/elfutils.cc',
'common/linux/file_id.cc',
'common/linux/guid_creator.cc',
'common/linux/linux_libc_support.cc',
'common/linux/memory_mapped_file.cc',
'common/linux/safe_readlink.cc'
])
libbreakpad_client_windows = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src'),[
'client/windows/crash_generation/client_info.cc',
'client/windows/crash_generation/crash_generation_client.cc',
'client/windows/crash_generation/crash_generation_server.cc',
'client/windows/crash_generation/minidump_generator.cc',
'client/windows/handler/exception_handler.cc',
'common/windows/guid_string.cc'
])
libbreakpad = builder.StaticLibraryProject('libbreakpad')
libbreakpad.sources = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src'),[
'processor/basic_code_modules.cc',
'processor/basic_source_line_resolver.cc',
'processor/call_stack.cc',
'processor/cfi_frame_info.cc',
'processor/convert_old_arm64_context.cc',
'processor/disassembler_x86.cc',
'processor/dump_context.cc',
'processor/dump_object.cc',
'processor/exploitability.cc',
'processor/exploitability_linux.cc',
'processor/exploitability_win.cc',
'processor/fast_source_line_resolver.cc',
'processor/logging.cc',
'processor/microdump.cc',
'processor/microdump_processor.cc',
'processor/minidump.cc',
'processor/minidump_processor.cc',
'processor/module_comparer.cc',
'processor/module_serializer.cc',
'processor/pathname_stripper.cc',
'processor/process_state.cc',
'processor/proc_maps_linux.cc',
'processor/simple_symbol_supplier.cc',
'processor/source_line_resolver_base.cc',
'processor/stack_frame_cpu.cc',
'processor/stack_frame_symbolizer.cc',
'processor/stackwalk_common.cc',
'processor/stackwalker.cc',
'processor/stackwalker_amd64.cc',
'processor/stackwalker_arm.cc',
'processor/stackwalker_arm64.cc',
'processor/stackwalker_address_list.cc',
'processor/stackwalker_mips.cc',
'processor/stackwalker_ppc.cc',
'processor/stackwalker_ppc64.cc',
'processor/stackwalker_riscv.cc',
'processor/stackwalker_riscv64.cc',
'processor/stackwalker_sparc.cc',
'processor/stackwalker_x86.cc',
'processor/symbolic_constants_win.cc',
'processor/tokenize.cc'
])
libbreakpad_linux = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src'),[
'common/linux/scoped_pipe.cc',
'common/linux/scoped_tmpfile.cc',
'processor/disassembler_objdump.cc'
])
libbreakpad_windows = []
libdisasm = builder.StaticLibraryProject('libdisasm')
libdisasm.sources = AddSourceFilesFromDir(os.path.join(builder.currentSourcePath, 'breakpad', 'src', 'third_party', 'libdisasm'),[
'ia32_implicit.c',
'ia32_insn.c',
'ia32_invariant.c',
'ia32_modrm.c',
'ia32_opcode_tables.c',
'ia32_operand.c',
'ia32_reg.c',
'ia32_settings.c',
'x86_disasm.c',
'x86_format.c',
'x86_imm.c',
'x86_insn.c',
'x86_misc.c',
'x86_operand_list.c'
])
libraries = [libbreakpad_client, libbreakpad, libdisasm]
libraries_platform_src = [(libbreakpad_client_linux, libbreakpad_client_windows), (libbreakpad_linux, libbreakpad_windows), ([],[])]
for cxx in Accelerator.targets:
for (library, (linux_src, windows_src)) in zip(libraries, libraries_platform_src):
binary = Accelerator.ConfigureLibrary(library, cxx, builder)
compiler = binary.compiler
# Wait for breakpad to be patched
compiler.sourcedeps += Accelerator.breakpad_patch
if compiler.target.platform == 'linux':
# Link against our zlib if linux
Accelerator.link_libz(cxx, builder)
binary.sources += linux_src
if compiler.target.platform == 'windows':
binary.sources += windows_src
compiler.cxxincludes += [
os.path.join(builder.currentSourcePath, 'breakpad', 'src')
]
compiler.defines += ['UNICODE']
Accelerator.libbreakpad = builder.Add(libbreakpad)
Accelerator.libbreakpad_client = builder.Add(libbreakpad_client)
Accelerator.libdisasm = builder.Add(libdisasm)

104
third_party/Configure vendored Normal file
View File

@ -0,0 +1,104 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os, sys
builder.SetBuildFolder('third_party/config')
for compiler in Accelerator.targets:
output = """
/* define if the compiler supports basic C++17 syntax */
#define HAVE_CXX17 1
/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1
/* Define to 1 if you have the <stdint.h> header file. */
#define HAVE_STDINT_H 1
/* Define to 1 if you have the <stdio.h> header file. */
#define HAVE_STDIO_H 1
/* Define to 1 if you have the <stdlib.h> header file. */
#define HAVE_STDLIB_H 1
/* Define to 1 if you have the <strings.h> header file. */
#define HAVE_STRINGS_H 1
/* Define to 1 if you have the <string.h> header file. */
#define HAVE_STRING_H 1
/* Define to 1 if you have the <unistd.h> header file. */
#define HAVE_UNISTD_H 1
"""
if compiler.target.platform == "linux":
output += """
/* Define to 1 if you have the <a.out.h> header file. */
#define HAVE_A_OUT_H 1
/* Define to 1 if you have the `getcontext' function. */
#define HAVE_GETCONTEXT 1
/* Define to 1 if you have the `getrandom' function. */
#define HAVE_GETRANDOM 1
/* Define to 1 if you have the <sys/mman.h> header file. */
#define HAVE_SYS_MMAN_H 1
/* Define to 1 if you have the <sys/random.h> header file. */
#define HAVE_SYS_RANDOM_H 1
/* Define to 1 if you have the <sys/stat.h> header file. */
#define HAVE_SYS_STAT_H 1
/* Define to 1 if you have the <sys/types.h> header file. */
#define HAVE_SYS_TYPES_H 1
/* Define to 1 if you have the `memfd_create' function. */
#define HAVE_MEMFD_CREATE 1
"""
if '-lrustc_demangle' in compiler.postlink:
output += """
/* Define to 1 if you have the `rustc_demangle' library (-lrustc_demangle). */
#define HAVE_LIBRUSTC_DEMANGLE 1
/* Define to 1 if you have the <rustc_demangle.h> header file. */
#define HAVE_RUSTC_DEMANGLE_H 1
"""
if '-pthread' in compiler.postlink:
output += """
/* Define if you have POSIX threads libraries and header files. */
#define HAVE_PTHREAD 1
"""
if '-lzstd' in compiler.postlink:
output += """
/* Define to 1 if you have the `zstd' library (-lzstd). */
#define HAVE_LIBZSTD 1
"""
output += """
/* Name of package */
#define PACKAGE "breakpad"
/* Define to the address where bug reports for this package should be sent. */
#define PACKAGE_BUGREPORT "google-breakpad-dev@googlegroups.com"
/* Define to the full name of this package. */
#define PACKAGE_NAME "breakpad"
/* Define to the full name and version of this package. */
#define PACKAGE_STRING "breakpad 0.1"
/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME "breakpad"
/* Define to the home page for this package. */
#define PACKAGE_URL ""
/* Define to the version of this package. */
#define PACKAGE_VERSION "0.1"
/* Version number of package */
#define VERSION "0.1"
"""
Accelerator.breakpad_config[compiler.target.arch] = [builder.AddOutputFile(os.path.join(compiler.target.arch, 'config.h'), output.encode())]

81
third_party/Patch vendored Normal file
View File

@ -0,0 +1,81 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import os, sys
import re
def Normalize(path):
return os.path.abspath(os.path.normpath(os.path.join(path)))
def AddFilesFromDir(path, files):
list = []
for file in files:
list.append(os.path.join(path, file))
return list
def AddCwdCommand(context, cwd, inputs,
argv,
outputs,
folder = -1,
dep_type = None,
weak_inputs = [],
shared_outputs = [],
env_data = None,
fake_output = None):
base_argv = [
sys.executable,
Normalize(os.path.join(context.currentSourcePath, 'cwd_cmd.py')),
cwd,
os.path.join(context.buildPath, 'third_party', fake_output) if fake_output else '_',
]
if fake_output :
outputs += [fake_output]
return context.AddCommand(inputs=inputs, argv=base_argv + argv, outputs=outputs, folder=folder, dep_type=dep_type, weak_inputs=weak_inputs, shared_outputs=shared_outputs, env_data=env_data)
git_dir = Normalize(os.path.join(builder.currentSourcePath, 'breakpad'))
patches = AddFilesFromDir(os.path.join(builder.currentSourcePath, '..', 'patches'), [
"0001-Ignore-invalid-modules-rather-than-bailing-on-the-en.patch",
"0002-Write-FUNC-records-instead-of-PUBLIC-for-ELF-symbols.patch",
"0003-Support-compilation-on-VS-2015.patch",
"0004-Fix-Linux-a-out-include.patch",
"0005-Add-LSS.patch"
])
# Reset the breakpad directory before applying patches
argv = [
'git',
'clean',
'-fdx',
'&&',
'git',
'reset',
'--hard'
]
git_reset = AddCwdCommand(builder, git_dir,
inputs=patches, # We want the reset to happen again if we modify the patch files
outputs=[],
fake_output='git_clean.txt',
argv=argv)
# Apply each of the patch available to the breakpad repository
argv = [
'git',
'apply',
'--reject'
]
git_patches = []
num = 0
for patch in patches:
patch_argv = argv + [patch]
git_patches += AddCwdCommand(builder, git_dir,
inputs=patches, # Redo the command if the patches changed
outputs=[],
fake_output=str(num) + '_patch.txt',
argv=patch_argv,
weak_inputs=git_reset if num == 0 else [git_patches[-1]] + git_reset # We need to for the git reset to be performed
)
num = num + 1
Accelerator.breakpad_patch = git_reset + git_patches

1
third_party/breakpad vendored Submodule

@ -0,0 +1 @@
Subproject commit 255dbbd061a400e7a3f601e82a62e65058b39e5e

21
third_party/cwd_cmd.py vendored Normal file
View File

@ -0,0 +1,21 @@
# vim: set ts=2 sw=2 tw=99 noet ft=python:
import subprocess, sys
argv = sys.argv[3:]
cwd = sys.argv[1]
print('CMD: ' + ' '.join(argv))
print('CWD: ' + cwd)
print('OUT: ' + sys.argv[2])
open(sys.argv[2], 'a').close()
args = { 'args': ' '.join(argv),
'stdout': subprocess.PIPE,
'stderr': subprocess.PIPE,
'shell': True,
'cwd': cwd }
p = subprocess.Popen(**args)
stdout, stderr = p.communicate()
print(stdout.decode("utf-8"))
if p.returncode != 0:
print(stderr.decode("utf-8"))
sys.exit(p.returncode)

1
third_party/zlib vendored Submodule

@ -0,0 +1 @@
Subproject commit d476828316d05d54c6fd6a068b121b30c147b5cd