My Library

• an API for git repositories
• Pure Haskell

Types

data GitObjectGeneric ref container entries
    = Blob {blobContent :: B.ByteString}
    | Tree {treeEntries :: container entries}
    | Commit
        { commitTreeRef    :: ref
        , commitParentRefs :: [ref]
        , commitAuthor     :: PersonTime
        , commitCommitter  :: PersonTime
        , commitSignature  :: Maybe B.ByteString
        , commitMessage    :: B.ByteString
        }
    | Tag
        { tagObjectRef  :: ref
        , tagObjectType :: B.ByteString
        , tagName       :: B.ByteString
        , tagTagger     :: PersonTime
        , tagAnnotation :: B.ByteString
        }
    deriving (Eq)

type GitObject = GitObjectGeneric Ref Set TreeEntry

Types

data TreeEntry = TreeEntry
    { entryPerms :: EntryPermission
    , entryName  :: B.ByteString
    , entryRef   :: Ref
    }

data PersonTime = PersonTime
    { personName :: B.ByteString
    , personMail :: B.ByteString
    , personTime :: B.ByteString
    , personTZ   :: B.ByteString
    }
    deriving (Eq)

data EntryPermission
    = Directory
    | Regular
    | Executable
    | SymbolicLink
    | SubModule
    deriving (Show, Eq)

type Ref  = B.ByteString
type Repo = FilePath

Functor

parseTree :: Parser GitObject
parseTree = Tree . fromList <$> many' parseTreeEntry

Applicative

parseMessage :: Parser B.ByteString
parseMessage = endOfLine *> takeByteString

Monad (Transformers)

type WithRepo = ReaderT Repo IO

hasLooseObject :: Ref -> WithRepo Bool
hasLooseObject ref = do
    path <- asks (sha1Path ref)
    liftIO $ doesFileExist path

Bit Twiddling

setMSBs :: (Bits t, Integral t) => [t] -> [Word8]
setMSBs [] = []
setMSBs (i:is) = map fromIntegral $ i : map setMSB is

Streaming

getNextEntry = do
    Just (Right tLen) <- PA.parse parseTypeLen
    baseRef <- case fst tLen of
        OfsDeltaObject -> do
            Just (Right offset) <- PA.parse parseOffset
            return $ encodeOffset offset
        RefDeltaObject -> do
            Just (Right ref)    <- PA.parse parseBinRef
            return $ fst $ decode ref
        _              -> return ""
    remainder <- get
    let decompressed = PZ.decompress' PZ.defaultWindowBits remainder
    PB.drawByte
    Just levelByte <- PB.peekByte
    let level = getCompressionLevel levelByte
    return (uncurry encodeTypeLen tLen, baseRef, decompressed, level)

Layout

src
├── Duffer
│   ├── Loose
│   │   ├── Objects.hs
│   │   └── Parser.hs
│   ├── Loose.hs
│   ├── Misc.hs
│   ├── Pack
│   │   ├── Entries.hs
│   │   ├── File.hs
│   │   └── Parser.hs
│   ├── Pack.hs
│   ├── Plumbing.hs
│   ├── Unified.hs
│   ├── WithRepo.hs
│   └── WorkObject.hs
└── Duffer.hs

Testing

• duffer uses its own repo to test itself.

Data Structures

• Merkle trees & Merkle DAGs
• Not too different from an efficiently implemented functional data structure.

Design

• Append-only object store and mutable ref store
• Separating the immutable and the mutable? Sounds like this language I know…

What’s next?

• A better API
• Multiple backends

Ideas

• A git-based project management system
• Using git as the backend of an application

Advice for beginners

• Don’t get too hung up on understanding monads before you write any code
• Lots of support on IRC and r/haskell
• IHaskell is a godsend and everyone should use it

Bad

• Documentation is mostly terrible
• Hard to find the right libraries
• Very demanding learning curve

Some thoughts

• I thought Haskell was pretty silly the first time I encountered it
• Only after a couple of bad production experiences did I see the point
• How else do we provide a compelling reason to learn it?

Historical Revisionism

• https://github.com/vaibhavsagar/git-internals-workshop

Git Objects as a Service

• https://github.com/vaibhavsagar/suppandi

Better Implementations

• https://github.com/vincenthz/hit - a pure Haskell implementation of Git objects
• https://github.com/jwiegley/gitlib - A type-safe interface with many backends
• http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/

Interesting Projects

• https://github.com/mirage/irmin - A distributed database with Git-like design

About this project

• Code is at https://github.com/vaibhavsagar/duffer
• Presentation is in the gh-pages branch or http://www.vaibhavsagar.com/duffer

RC

https://www.recurse.com