Wednesday, December 26, 2012

Episode 23: Hadoop


Download

News
Tool of the Show
Book of the Show


Hadoop

History
  • Jeff Dean & Sanjay Ghemawat wrote the paper MapReduce
  • Created by Doug Cutting while he was at Yahoo!.
  • Intended to support Lucene (search engine reverse indexing).
  • Facebook announces their hadoop filesystem has grown to 100 petabytes. 
Features
  • HDFS: Hadoop Distributed Filesystem
  • HBase: A distributed, column-oriented database
  • Zookeeper: Distributed coordination service
  • Crunch: Simplified API for creating mapreduce pipelines.

        Strengths
        • Scale-free
        • Fault Tolerant
        • Can add/remove hardware in real-time.
        Weaknesses
        • Long spin up / spin down time.
          • Worker Pools
        • Excessive Serialization/deserialization
        • Excessive Materialization

        Tools
        • Avro: A serialization framework
        • Pig & Hive: querying and storing large datasets

        Uses
        • Storing/Manipulating Big Data.

        Sunday, December 9, 2012

        Episode 22: LaTeX


        Download

        News
        Tool of the Show
        Book of the Show
        LaTeX

        History
        • TeX
          • Created by Donald Knuth
          • Based on 19th century typesetting machines
        • LaTeX
          • Created by Leslie Lamport
          • TeX with many macros

            Features
            • bibtex: bibliography management
              • Most citations have a .bib file for easy import
            • Decouple the content (the source) from the style
              • Turn your paper into a book, book into a webpage, etc.
            • Automatic generation of references, table of contents, bibliography
            • Export to many formats
              • dvi (default)
              • latex2html
              • pdf (using pdflatex)
              • ps
            • Not WYSIWYG, WYSIWYM

                  Tools

                  Uses
                  • Academic writing
                    • Conferences, journals
                  • Writing Textbooks
                  • Resumes, curricula vitae