Limited Offer Get 25% off — use code BESTW25
No AI No Plagiarism On-Time Delivery Free Revisions
Claim Now
Prices start at $5 per page · plagiarism-free papers · free revisions for 14 days 24/7 support: support@bestwriters.com
Uncategorized

DATA WAREHOUSE IMPLEMENTATION

  • Posted
  • Authoradmin
  • Length803 words
  • Reading time4 min
Write My Assignment FREE

Efficient Data Cube Computation
 Data cube can be viewed as a lattice of cuboids
 The bottom-most cuboid is the base cuboid
 The top-most cuboid (apex) contains only one cell
 How many cuboids in an n-dimensional cube with L
levels?
 Materialization of data cube
 Materialize every (cuboid) (full materialization), none (no
materialization), or some (partial materialization)
 Selection of which cuboids to materialize
 Based on size, sharing, access frequency, etc.

Cube Operation
 Cube definition and computation in DMQL
define cube sales[item, city, year]: sum(sales_in_dollars)
compute cube sales
 Transform it into a SQL-like language (with a new operator
cube by, introduced by Gray et al.’96)
SELECT item, city, year, SUM (amount)
FROM SALES
CUBE BY item, city, year
 Need compute the following Group-Bys
(date, product, customer),
(date,product),(date, customer), (product, customer),
(date), (product), (customer)

Cube Computation: ROLAP-Based Method
 Efficient cube computation methods
 ROLAP-based cubing algorithms (Agarwal et al’96)
 Array-based cubing algorithm (Zhao et al’97)
 Bottom-up computation method (Beyer & Ramarkrishnan’99)
 H-cubing technique (Han, Pei, Dong & Wang:SIGMOD’01)
 ROLAP-based cubing algorithms
 Sorting, hashing, and grouping operations are applied to the
dimension attributes in order to reorder and cluster related
tuples
 Grouping is performed on some sub-aggregates as a “partial
grouping step”
 Aggregates may be computed from previously computed
aggregates, rather than from the base fact table
42
Cube Computation: ROLAP-Based Method (2)
 This is not in the textbook but in a research paper
 Hash/sort based methods (Agarwal et. al. VLDB’96)
 Smallest-parent: computing a cuboid from the
smallest, previously computed cuboid
 Cache-results: caching results of a cuboid from which
other cuboids are computed to reduce disk I/Os
 Amortize-scans: computing as many as possible
cuboids at the same time to amortize disk reads
 Share-sorts: sharing sorting costs cross multiple
cuboids when sort-based method is used
 Share-partitions: sharing the partitioning cost across
multiple cuboids when hash-based algorithms are used
43
Multi-way Array Aggregation for Cube
Computation
 Partition arrays into chunks (a small subcube which fits in memory).
 Compressed sparse array addressing: (chunk_id, offset)
 Compute aggregates in “multiway” by visiting cube cells in the order which
minimizes the # of times to visit each cell, and reduces memory access and
storage cost.

Multi-Way Array Aggregation for
Cube Computation (Cont.)
 Method: the planes should be sorted and computed
according to their size in ascending order.
 See the details of Example 2.12 (pp. 75-78)
 Idea: keep the smallest plane in the main memory,
fetch and compute only one chunk at a time for the
largest plane
 Limitation of the method: computing well only for a small
number of dimensions
 If there are a large number of dimensions, “bottomup
computation” and iceberg cube computation
methods can be explored
47
Indexing OLAP Data: Bitmap Index
 Index on a particular column
 Each value in the column has a bit vector: bit-op is fast
 The length of the bit vector: # of records in the base table
 The i-th bit is set if the i-th row of the base table has the value for
the indexed column
 not suitable for high cardinality domains

Indexing OLAP Data: Join Indices
 Join index: JI(R-id, S-id) where R (R-id, …)   S
(S-id, …)
 Traditional indices map the values to a list of
record ids
 It materializes relational join in JI file and
speeds up relational join — a rather costly
operation
 In data warehouses, join index relates the values
of the dimensions of a start schema to rows in
the fact table.
 E.g. fact table: Sales and two dimensions city
and product
 A join index on city maintains for each
distinct city a list of R-IDs of the tuples
recording the Sales in the city
 Join indices can span multiple dimensions

Efficient Processing OLAP Queries
 Determine which operations should be performed on the
available cuboids:
 transform drill, roll, etc. into corresponding SQL and/or
OLAP operations, e.g, dice = selection + projection
 Determine to which materialized cuboid(s) the relevant
operations should be applied.
 Exploring indexing structures and compressed vs. dense
array structures in MOLAP
50
Metadata Repository
 Meta data is the data defining warehouse objects. It has the following
kinds
 Description of the structure of the warehouse
 schema, view, dimensions, hierarchies, derived data defn, data mart
locations and contents
 Operational meta-data
 data lineage (history of migrated data and transformation path),
currency of data (active, archived, or purged), monitoring information
(warehouse usage statistics, error reports, audit trails)
 The algorithms used for summarization
 The mapping from operational environment to the data warehouse
 Data related to system performance
 warehouse schema, view and derived data definitions
 Business data
 business terms and definitions, ownership of data, charging policies
51
Data Warehouse Back-End Tools and Utilities
 Data extraction:
 get data from multiple, heterogeneous, and external
sources
 Data cleaning:
 detect errors in the data and rectify them when possible
 Data transformation:
 convert data from legacy or host format to warehouse
format
 Load:
 sort, summarize, consolidate, compute views, check
integrity, and build indicies and partitions
 Refresh
 propagate the updates from the data sources to the
warehouse

The post DATA WAREHOUSE IMPLEMENTATION appeared first on My Assignment Online.

Plagiarism Free Assignment Help

Expert Help With This Assignment — On Your Terms

✓ Native UK, USA & Australia writers ✓ Deadline from 3 hours ✓ 100% Plagiarism-Free — Turnitin included ✓ Unlimited free revisions ✓ Free to submit — compare quotes

admin

Academic writer and editor at BestWriters.

Get your paper off your plate tonight

Post your instructions for free. Prices start at $5 per page.

Place an order