当前位置: 首页 > 科技观察

哨兵监控-Snuba数据中心本地开发环境配置

时间:2023-03-14 23:29:09 科技观察

本文转载自微信公众号《黑客下午茶》,作者为邵。转载本文请联系黑客下午茶公众号。克隆仓库分别克隆getentry/sentry和getsentry/snuba:gitclonehttps://github.com/getsentry/sentry.gitclonehttps://github.com/getsentry/snuba.git安装系统依赖(以Mac为一个例子)XcodeCLItoolsxcode-select--installBrewfile进入sentry文件夹,你会看到一个Brewfile文件:cdsentryBrewfile#requiredtorundevservicescask'docker'brew'pyenv'#requiredforpyenv'spython-buildbrew'openssl'brew'readline'#requiredforyarntest-ubrew'守望者'#requiredtobuildsomeofsentry'sdependenciesbrew'pkgconfig'brew'libxslt'brew'libxmlsec1'brew'geoip'#CurrentlyneededbecauseonBigSurthere'ssnowheelforitbrew'librdkafka'#direnvisn'tdefinedhere,becausewehaveitconfiguredtocheckforabootstrappedenvironment.#Ifit'sinstalledintheearlystepsofthealvthesetupprocess,ithomebrew/cask'#requiredforacceptancetestingcask'chromedriver'如果你本地安装了DockerDesktop并启动了,可以注释掉cask'docker'。接下来运行:brewbundle--verbose如果之前本地没有DockerDesktop,需要手动启动:open-g-aDocker.appbuildtoolchainSentrydependsonPythonWheels(packagescontainingbinaryextensionmodules),官方分布式适用于以下平台:符合PEP-513的Linux(manylinux1)macOS10.15或更高版本如果您的开发机器未运行上述系统之一,则需要安装Rust工具链。按照https://www.rust-lang.org/tools/install中的说明安装编译器和相关工具。安装后,Sentry安装程序将自动使用Rust构建所有二进制模块,无需额外配置。官方通常会跟踪最新的Rust稳定版本,每六周更新一次。因此,请确保偶尔运行Rust工具链是最新的:rustupupdatestablePythonSentry使用pyenv来安装和管理Python版本。它在您运行brewbundle时安装。要安装所需版本的Python,您需要运行以下命令。这需要一段时间,因为您的计算机实际上正在编译Python!makesetup-pyenv假设你是这里的Zsh用户。如果你输入whichpython你应该看到类似$HOME/.pyenv/shims/python而不是/usr/bin/python的东西。这是因为以下内容已添加到您的启动脚本中:cat~/.zprofile并且您将看到类似这样的内容:#MacPortsInstalleradditionon2021-10-20_at_11:48:22:addinganappropriatePATHvariableforusewithMacPorts.exportPATH="/opt/local/bin:/opt/local/sbin:$PATH"#FinishedadaptingyourPATHenvironmentvariableforusewithMacPorts.#ItisassumedthatpyenvisinstalledviaBrew,sothisisallweneedtodo.eval"$(pyenvinit--path)"虚拟环境你现在可以创建你的Python虚拟环境了。运行:python-mvenv.venv并激活虚拟环境:source.venv/bin/activate如果一切正常,运行whichpython现在应该会产生类似于/Users/you/sentry/.venv/bin/python的结果。Snuba配置实战启动Snuba相关依赖项容器cd../sentrygitcheckoutmastergitpullsource.venv/bin/activatesentrydevservicesup--exclude=snuba#11:17:59[WARNING]sentry.utils.geo:settings.GEOIP_PATH_MMDB1:notconfigured[INFO]sentry.plugins.github:apps-not-configured#>Pullingimage'postgres:9.6-alpine'#>Pullingimage'yandex/clickhouse-server:20.3.9.70'#>Notstartingcontainer'sentry_relay'因为它应该按需启动与devserver。#>Creating'sentry_redis'volume#>Creating'sentry_zookeeper_6'volume#>Creating'sentry_kafka_6'volume#>Creatingcontainer'sentry_redis'#>Creatingcontainer'sentry_zookeeper'#>Creatingcontainer'sentry_kafka'#>Startingcontainer'sentry_redis'(listening:('127.0.0.1',6379))#>Startingcontainer'sentry_kafka'(listening:('127.0.0.1',9092))#>Startingcontainer'sentry_zookeeper'#>Creating'sentry_clickhouse'volume#>Creatingcontainer'sentry_clickhouse'#>Creating'sentry_postgres'volume#>正在创建'sentry_wal2json'volume#>Startingcontainer'sentry_clickhouse'(listening:('127.0.0.1',9000),('127.0.0.1',9009),('127.0.0.1',8123))#>Creatingcontainer'sentry_postgres'#>Startingcontainer'sentry_postgres'(listening:('127.0.0.1',5432))这将在master上获取最新版本的哨兵,并调出所有snuba依赖项。Snuba主要依赖clickhouse、zookeeper、kafka、redis相关容器。用dockerps检查:1149a6f6ff23postgres:9.6-alpine"docker-entrypoint.s..."3minutesagoUp3minutes127.0.0.1:5432->5432/tcpsentry_postgresa7f3af7d52bbyandex/clickhouse-server:20.3.9.70"/entrypoint.sh"3minutesagoUp1.sh“3minutesagoUp1.7:8123->8123/tcp,127.0.0.1:9000->9000/tcp,127.0.0.1:9009->9009/tcpsentry_clickhouse68913ee15c43confluentinc/cp-zookeeper:6.2.0”/etc/confluent/dock..."3minutesagoUp3minutes2181//tcp,3888/tcpsentry_zookeeper5a248eb26ed3confluentinc/cp-kafka:6.2.0"/etc/confluent/dock..."3minutesagoUp3minutes127.0.0.1:9092->9092/tcpsentry_kafka0573aff7b5afredis-points.7"0go2.7.1:6379->6379/tcpsentry_redis设置Snuba虚拟环境cdsnubamakepyenv-setuppython-mvenv.venvsource.venv/bin/activatepipinstall--upgradepip==21.1.3makedevelop查看迁移列表snubamigrationslist#system#[]0001_migrations##events#[]0001_events_initial#[]0002_events_onpremise_compatibility#[]0003_errors#[]0004_errors_onpremise_compatibility#[]0005_events_tags_hash_map(blocking)#[]0006_errors_tags_hash_map(blocking)#[]0007_groupedmessages#[]0008_groupassignees#[]0009_errors_add_http_fields#[]0010_groupedmessages_onpremise_compatibility(blocking)#[]0011_rebuild_errors#[]0012_errors_make_level_nullable#[]0013_errors_add_hierarchical_hashes#[]0014_backfill_errors(blocking)#[]0015_truncate_events##transactions#[]0001_transactions#[]0002_transactions_onpremise_fix_orderby_and_partitionby(blocking)#[]0003_transactions_onpremise_fix_columns(blocking)#[]0004_transactions_add_tags_hash_map(blocking)#[]0005_transactions_add_measurements#[]0006_transactions_add_http_fields#[]0007_transactions_add_discover_cols#[]0008_transactions_add_timestamp_index#[]0009_transactions_fix_title_and_message#[]0010_transactions_nullable_trace_id#[]0011_transactions_add_span_op_##breakdowns#[]0add_1#spancover_transactiondiscover_merge_table#[]0002_discover_add_deleted_tags_hash_map#[]0003_discover_fix_user_column#[]0004_discover_fix_title_and_message#[]0005_discover_fix_transaction_name#[]0006_discover_add_trace_id#[]0007_discover_add_span_id##outcomes#[]0001_outcomes#[]0002_outcomes_remove_size_and_bytes#[]0003_outcomes_add_category_and_quantity#[]0004_outcomes_matview_additions(blocking)##metrics#[]0001_metrics_buckets#[]0002_metrics_sets#[]0003_counters_to_buckets#[]0004_metrics_counters#[]0005_metrics_distributions_buckets#[]0006_metrics_distributions#[]0007_metrics_sets_granularity_10#[]0008_metrics_counters_granularity_10#[]0009_metrics_distributions_granularity_10#[]0010_metrics_sets_granularity_1h#[]0011_metrics_counters_granularity_1h#[]0012_metrics_distributions_granularity_1h#[]0013_metrics_sets_granularity_1d#[]0014_metrics_counters_granularity_1d#[]0015_metrics_distributions_granularity_1d##sessions#[]0001_sessions#[]0002_sessions_aggregates#[]0003_sessions_matview运行迁移snubamigrationsmigrate--force#......#2021-12-0119:45:57,557Runningmigration:0014_metrics_counters_granularity_1d#2021-12-0119:45:57,575Finished:0014_metrics_counters_granularity57,1:245-9111Runningmigration:0015_metrics_distributions_granularity_1d#2021-12-0119:45:57,610Finished:0015_metrics_distributions_granularity_1d#2021-12-0119:45:57,623Runningmigration:0001_sessions#2021-12-0119:45:57,656Finished:0001_sessions#2021-12-0119:45:57,669Runningmigration:0002_sessions_aggregates#2021-12-0119:45:57,770Finished:0002_sessions_aggregates#2021-12-0119:45:57,792Runningmigration:0003_sessions_matview#2021-12-0119:45:57,849Finished:0003_sessions_matview#Finishedrunningmigrations检查迁移进入Clickhouse容器:dockerexec-itsentry_clickhouseclickhouse-client#运行如下`sql`语句:selectcount()fromsentry_local#ClickHouseclientversion20.3.9.70(officialbuild).#Connectingtolocalhost:9000asuserdefault.#ConnectedtoClickHouseserverversion20.3.9revision54433.#a7f3af7d52bb:)selectcount()fromsentry_local#SELECTcount()#FROMsentry_local#┌─count()─┐#│0│#└───────────┘#1rowsinset.Elapsed:0.008sec.#a7f3af7d52bb:)查看相关实体数据集snubaentitieslist#DeclaredEntities:#discover#events#groups#groupassignee#groupedmessage#metrics_sets#metrics_counters#metrics_distributions#outcomes#outcomes_raw#sessions#org_sessions#spans#transactions#discover_transactions#discover_events命令将启动开发服务器api和所有Snuba消费者从Kafka摄取数据:snubadevserver转到http://localhost:1218/events/snql你会看到一个简单的查询UI