Opened 8 months ago

Last modified 3 weeks ago

#8545 new task

Reorganize Git repositories

Reported by: hvr Owned by: hvr
Priority: normal Milestone: 7.10.1
Component: Trac & Git Version:
Keywords: git Cc: thoughtpolice
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets: #8251

Description

This is to track the progress for the GitRepoReorganization which is to be tackled post-7.8

Change History (16)

comment:1 Changed 6 months ago by Herbert Valerio Riedel <hvr@…>

In 5f54d67818ee7a74325eed130438beba96510e43/ghc:

Update `sync-all` and others files w.r.t. merged testsuite (re #8545)

See merge commit 66693401b98cb5aa912948af7bbd2182474f50c4

This commit also adds a check for a left-over testsuite/.git folder to sync-all

This way, the first time sync-all is called after updating to a
post-testsuite-merge (see #8545) state of ghc.git, the sync-all script
aborts with an error message if a `testsuite/.git` folder is detected and
thus forces the user to take action.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:2 Changed 6 months ago by Herbert Valerio Riedel <hvr@…>

In 66693401b98cb5aa912948af7bbd2182474f50c4/ghc:

Fold testsuite.git into ghc.git (re #8545)

This commit performs a subtree merge of testsuite.git into ghc.git;
The next commit will adapt `sync-all` et al. to the new situation.

At the time of merge, testsuite.git was at commit
[998a816ae89c4fd573f4abd7c6abb346cf7ee9af/testsuite]

The following steps have been used to accomplish this merge:

1. Clone a fresh testsuite.git copy (& cd into)

2. Remove accidentally committed binary files from history

 git filter-branch \
  --index-filter "git rm -r --cached --ignore-unmatch \
    tests/haddock/should_compile_flag_nohaddock/a.out \
    tests/haddock/should_compile_noflag_nohaddock/a.out \
    tests/ghc-regress/haddock/should_compile_flag_nohaddock/a.out \
    tests/ghc-regress/haddock/should_compile_noflag_nohaddock/a.out \
    tests/ghc-regress/dph/diophantine/dph-diophantine-fast \
    tests/ghc-regress/dph/diophantine/dph-diophantine-opt \
    tests/ghc-regress/dph/primespj/dph-primespj-fast \
    tests/ghc-regress/dph/quickhull/dph-quickhull-fast \
    tests/ghc-regress/dph/smvm/dph-smvm \
    tests/ghc-regress/dph/sumnats/dph-sumnats \
    tests/ghc-regress/dph/words/dph-words-fast \
    tests/ghc-regress/plugins/plugins01" \
   HEAD

3. Rename all paths in testsuite.git to be prefixed with `testsuite/`

 git filter-branch -f --prune-empty --tree-filter \
  "mkdir -p testsuite; \
   git ls-tree --name-only \$GIT_COMMIT | xargs -I files mv files testsuite/"

4. cd into ghc/ checkout, and perform subtree merge of testsuite into ghc
   (see also http://nuclearsquid.com/writings/subtree-merging-and-you/)

 cd ../ghc/
 git remote add -f testsuite ../testsuite/.git
 git merge -s ours --no-commit testsuite/master
 git read-tree --prefix=/ -u testsuite/master
 git commit

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:3 Changed 6 months ago by hvr

Note: While the mail commit notification hook was disabled (and before re-enabling it, the commit-id database updated to contain the new commits), the Trac update hook wasn't from the beginning of the transaction; so about 80 trac updates were performed on ancient tickets (+ mail notifications); this really needs to be taken into account next time.

comment:4 Changed 5 months ago by Herbert Valerio Riedel <hvr@…>

In ad44e47542a822ac3e02cf514b5d2be52880fc95/ghc:

Switch to relative URLs in .gitmodules

Previously, the `http://`-protocol part was hardcoded in the URLs, causing
the initial clone process to fall back to `http://` even when the ghc.git repo
was cloned via one of the other 3 supported transport protocols.

This is slightly related to #8545, as it will make it possible to e.g.

    git clone --recursive git://git.haskell.org/ghc

and clone ghc.git including all submodules in one go (i.e. w/o `sync-all`),
and w/o falling back to a different (hardwired) Git transport protocol for
the submodules.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:5 Changed 4 months ago by Herbert Valerio Riedel <hvr@…>

In 34b072177b687c8fcc24f87293beae0752e82d32/ghc:

Convert haddock into a proper submodule (re #8545)

This should help contribute content to

  https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git/Submodules

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:6 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In a6e68af1aba7294a18ecb15784d9fe5d90289101/ghc:

Fold ghc-prim.git into ghc.git (re #8545)

At the time of merge, ghc-prim.git was at
[ad9bf96815cb5a9bb4acc51c99eff20be3e50da3/ghc-prim]

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:7 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In c83bec7853e8f59cb2c5869baf2d2e0179c6c195/ghc:

Fold base.git into ghc.git (re #8545)

At the time of merge, base.git was at
[52c0b09036c36f1ed928663abb2f295fd36a88bb/base]

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:8 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In 85febc048d733eb5c8a942e505af26156295f3a6/ghc:

Fold integer-simple.git into ghc.git (re #8545)

At the time of merge, integer-simple.git was at
[9e8b924f68c4cdb6c7ae88f274baa3560aaa305e/integer-simple]

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:9 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In 670599db0df700317eb42ac4f42a841c78ffef5d/ghc:

Fold integer-gmp.git into ghc.git (re #8545)

At the time of merge, integer-gmp.git was at
[d7bff4dddfa10389156ca11f75a5a23e78cf3ab0/integer-gmp]

Note: All but the last/current GMP tarball were removed from the
      history to keep the Git history size at a minimum.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:10 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In 8bcb2068e496b36d74c655c868cc93be2de45d94/ghc:

Fold template-haskell.git into ghc.git (re #8545)

At the time of merge, template-haskell.git was at
[9bcc122819a6f4a2ae7ad569717324b8368e801c/template-haskell]

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:11 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In 41f5b7e3e0648302b9c5dc485917a391d21d15a1/ghc:

Update `sync-all` and `packages` wrt to fold-in

This adapts `sync-all` and `packages` to the recently folded-in Git
repositories

 - `base.git`
 - `ghc-prim.git`
 - `integer-gmp.git`
 - `integer-simple.git`
 - `template-haskell.git`

See #8545 as well as 5f54d67818ee7a74325eed130438beba96510e43 for more
details.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:12 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In 386e874e365e310214b701e6f3cb74b4f75348f4/ghc:

Update Cabal source-repository entries (re #8545)

This adapts the source-repository entries to match the new situation of
base.git, ghc-prim.git, integer-gmp.git, integer-simple.git, and
template-haskell.git being folded into ghc.git

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:13 Changed 3 months ago by Herbert Valerio Riedel <hvr@…>

In bb857596d1f09260908a0dd4f05cc25c2e822c4c/ghc:

Adapt .gitignore (re #8545)

This adapts the top-level .gitignore file to match the new situation of
base.git, ghc-prim.git, integer-gmp.git, integer-simple.git, and
template-haskell.git being folded into ghc.git

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:14 Changed 3 months ago by hvr

In 33e585d6eacae19e83862a05b650373b536095fa/ghc:

(The changeset message doesn't reference this ticket)

(this was wrongly filed in #9016)

comment:15 Changed 3 weeks ago by Herbert Valerio Riedel <hvr@…>

In db19c665ec5055c2193b2174519866045aeff09a/ghc:

Convert loose sub-repos into proper submodules (re #8545)

Specifically, the following sub-repos/modules are converted:

 - libffi-tarballs
 - libraries/array
 - libraries/deepseq
 - libraries/directory
 - libraries/dph
 - libraries/filepath
 - libraries/haskell2010
 - libraries/haskell98
 - libraries/hoopl
 - libraries/hpc
 - libraries/old-locale
 - libraries/old-time
 - libraries/parallel
 - libraries/process
 - libraries/stm
 - libraries/unix
 - nofib
 - utils/hsc2hs

N.B. ghc-tarballs is not converted as it will probably be handled
     differently in the future.

Signed-off-by: Herbert Valerio Riedel <hvr@gnu.org>

comment:16 Changed 3 weeks ago by Thomas Miedema <thomasmiedema@…>

In 72fe49d88565fc1dd807a0e185a5aa6fc4989ea0/ghc:

sync-all: infer remotepath from .gitmodules file

After this commit, running `sync-all remote set-url` works properly for the
haddock package:

    command: ./sync-all -r git://git.haskell.org remote set-url origin
    before: git://git.haskell.org/packages/haddock.git
    after: git://git.haskell.org/haddock.git

By doing the `remotepath` lookup before the `$is_github_repo` check,
running `sync-all remote set-url` now also works properly for submodule
repos on github:

    command: ./sync-all -r git://github.com/ghc remote set-url origin
    before: git://github.com/ghc/packages/binary
    after: git://github.com/ghc/packages-binary

* Relevant prior commits:
4f4357 "Make `sync-all remote set-url` use normalized `/packages/` urls"
34b072 "Convert haddock into a proper submodule (re #8545)"
974a97 "sync-all: Apply submodule url rewriting also to stuff in util/"
Note: See TracTickets for help on using tickets.