课程文献中心 更多>>

关键词： 计算机   大数据 Hadoop Hive 数据科学  

ISBN: (纸本)9787115526090

摘要： 本书以Hive为开发平台，主要介绍了如何使用HiveQL来查询和分析存储在Hadoop分布式文件系统上的大数据集合，具体内容包括Hive入门、Hive数据库及表操作、Hive元数据、Hive高级操作、Hive函数与Streaming、Hive视图与索引、Hive调优、Hive与HBase集成、数据迁移框架Sqoop等。本书介绍的每个任务都运用了大量案例，紧密结合实际应用，融入了含金量十足的开发经验。在此基础上，本书通过丰富的练习和操作实践，帮助读者巩固所学的内容。本书配以多元的学习资源和支持服务，包括视频、案例素材、学习社区等，为读者提供全方位的学习体验。 本书适合作为计算机、大数据等相关专业的教材，也适合具有一定Linux或Java开发基础且想从事大数据开发的人员阅读学习，还可以作为大数据分析与运维人员的参考用书。

关键词： 数据仓库   ETL   元数据   数据清洗  

摘要： 针对之前数据抽取任务的零散、串行、任务多而难管理。设计实现一种多数据流并行抽取数据,对异构平台及各种类型的数据源进行多数据流并行抽取,扩展了ETL(Extract,Transform and Load)现有的系统架构,为今后对海量数据的处理提供了统一高效的数据来源。

关键词： 不动产登记信息   数据仓库   共享模式   国密算法   电子证照  

摘要： 本文提供了省级不动产登记信息共享和安全防护的解决方案。不同的共享服务对象具有不同的不动产信息共享范围、方式和安全要求。针对不同共享对象和场景,设计了纵向联通和横向信息共享的两种模式,围绕信息平台、数据存储和共享模式提出了实现思路,并提供了信息汇聚、上交、共享环节的安全防护解决方案。通过实践运行,解决方案取得较好效果。

关键词： 工作流   Activiti5   定时任务   数据仓库   推送技术  

摘要： 本文介绍并分析了Activiti5工作流管理平台的总体设计,在此基础上,研究了工作流任务定时的设计与实现。利用定时器边界事件实现定时任务,利用DWR推送技术实现刷新,该方案在教学训练系统中得到了应用,收到了良好的效果。

关键词： Data management   Web portal   Observation data   Monitoring   R-Shiny   Ecosystem developments   ELECTROMAGNETIC INDUCTION   SOIL-SALINITY   WATER  

摘要： Exploring hydrological and ecological processes plays a key role in understanding ecosystem developments. In this respect, the constructed catchment, Chicken Creek, has been established for fundamental and interdisciplinary scientific research. Since 2005, an ongoing monitoring program has been launched to measure hydrological, biological, meteorological, and pedological parameters during the ecological development of the site. This comprehensive and multidisciplinary monitoring program has produced a diverse large data set. Handling such complex data for research purposes can be a cumbersome task. Consequently, we developed an online data portal () to efficiently handle the data from Chicken Creek catchment. The portal was constructed using Shiny package of the R programming language. This platform provides a web-based data repository allowing for data discovery, download, visualization, and analysis. The data include time series of different parameters from installed sensors, data from laboratory analyses, vegetation data, data from sampling campaigns, and aerial photos. This platform demonstrates the relevancy and potentiality of the R-Shiny for constructing an online data portal to be used for multidisciplinary scientific purposes. The Chicken Creek data portal thus provides a comprehensive and reliable database to give scientists a fast and easy access to all collected data.

关键词： Administrative data   data linkage   injury prevention   motor vehicle crash   CRASH   RATES  

摘要： Objective: Our objective is to describe the development of the New Jersey Safety and Health Outcomes (NJ-SHO) data warehouse, a unique and comprehensive data source that integrates various state-level administrative databases in New Jersey to enable the field of traffic safety to address critical, high-priority research questions. Methods: We have obtained full identifiable data from the following statewide administrative databases for the state of New Jersey: (1) driver licensing database; (2) Administration Office of the Courts data on traffic-related citations; (3) police-reported crash database; (4) birth certificate data; (5) death certificate data; and (6) hospital discharge data as well as (7) childhood electronic records from New Jersey residents who were patients of the Children's Hospital of Philadelphia pediatric health care network and (8) census tract-level indicators. We undertook an iterative process to develop a linkage algorithm in LinkSolv 9.0 software using records for individuals born in select birth years (1987 and 1988) and subsequently execute the linkage for the entire study period (2004-2017). Several metrics were used to evaluate the quality of the linkage process. Results: We identified a total of 62,685,619 records and 19,247,363 distinct individuals; 10,352,998 of these individuals had more than one record brought together during the linkage process. Our evaluation of this linkage suggests that the linkage was of high quality. Conclusions: The resulting NJ-SHO data warehouse will be one of the most comprehensive and rich traffic safety data warehouses to date. The warehouse has already been utilized for numerous studies and will be fully primed to support a host of rigorous studies, both in and beyond the field of traffic safety.

关键词： 医院管理   决策系统   数据仓库  

摘要： 目的:为院长正确决策,医疗质量评价,医院等级评审,医院管理流程优化等提供数据支持。方法:通过ETL,CDC等多种技术抽取生产数据库的数据,构建基于Oracle数据库的数据仓库。然后基于数据仓库的数据进行分析,对生产数据库和在线业务系统不产生影响。结果及结论:医院决策支持系统的研究和实现将改变医院管理现状,提高医院管理水平和效率,促进医院健康发展。

关键词： Health services research   Pediatrics   General surgery   ASSOCIATION   OUTCOMES  

摘要： Background: Given the rarity of pediatric surgical disease, it is important to consider available large-scale data resources as a means to better study and understand relevant disease-processes and their treatments. The Military Health System Data Repository (MDR) includes claims-based information for >3 million pediatric patients who are dependents of members and retirees of the United States Armed Services, but has not been externally validated. We hypothesized that demographics and selected outcome metrics would be similar between MDR and the previously validated American College of Surgeons National Surgical Quality Improvement Program-Pediatric (NSQIP-P) for several common pediatric surgical operations. Methods: We selected five commonly performed pediatric surgical operations: appendectomy, pyeloplasty, pyloromyotomy, spinal arthrodesis for scoliosis, and facial reconstruction for cleft palate. Among children who underwent these operations, we compared demographics (age, sex, and race) and clinical outcomes (length of hospital stay [LOS] and mortality) in the MDR and NSQIP-P, including all available overlapping years (2012-2014). Results: Age, sex, and race were generally similar between the NSQIP-P and MDR. Specifically, these demographics were generally similar between the resources for appendectomy (NSQIP-P, n=20,602 vs. MDR, n=4363; median age 11 vs. 12years; female 40% vs. 41%; white 75% vs. 84%), pyeloplasty (NSQIP-P, n=786 vs. MDR, n=112; median age 0.9 vs. 2years; female 28% vs. 28%; white 71% vs. 80%), pyloromyotomy, (NSQIP-P, n=3827 vs. MDR, n=227; median age 34 vs. <1year, female 17% vs. 16%; white 76% vs. 89%), scoliosis surgery (NSQIP-P, n=5743 vs. MDR, n=95; median age 14.2 vs. 14years; female 75% vs. 67%; white 72% vs. 75%), and cleft lip/palate repair (NSQIP-P, n=6202 vs. MDR, n=749; median age, 1 vs. 1year; female 42% vs. 45%; white 69% vs. 84%). Length of stay and 30-day mortality were similar between resources. LOS and 30-day mortality were

关键词： 数据仓库   实时数据分析   敏捷数仓   虚拟模型  

摘要： 传统数据仓库在当前时代背景下体现出一些疲态。论文提出了在高算力平台支持下新的敏捷型数据仓库建设体系,该体系建立在数据仓库维度建模方法论之上,以内存计算为技术基础,以业务分析矩阵方法为实施指导路线,自下而上、循序渐进地构建企业级数据仓库体系,有利于实现端到端实时分析、更加适应需求变化。该体系被应用于某大型能源行业央企,取得了显著效果。

关键词： 数据仓库   聚类分析   参数筛选   高炉生产   参数优化  

摘要： 为了更精准地评判高炉运行情况,量化高炉指导方针。根据高炉生产过程的特点,应用大数据技术对高炉生产参数与铁水产量和高炉能耗等指标进行数据驱动分析,提出了一种优化高炉生产参数的新方法。首先,对某钢铁厂高炉各工序历史数据进行了采集、清洗、过滤和整合,建立了高炉数据仓库。然后,将多种聚类算法相结合,完成了对高炉炉况变化的详细划分。运用工艺经验与递归特征消除算法相结合,全面筛选得到能够反映炉况波动的强相关变量。应用统计学方法分析得出Class_a炉况对应的核心参数的最优范围,这对指导现场生产、保持高炉长期稳定顺行具有重要意义。