FAQ
I think dropping the caches puts the tests further from real world usage,
especially if my editor is calling goimports every time I save, but just
for in case, here are the execution times dropping the fs cache:

goimports (current implementation):
real 0m6.288s

goimports (with cache, cold run with cache not built yet):
real 0m6.961s

goimports (with cache populated):
real 0m0.110s


That's a design problem with tools/import. It shouldn't need to walk
everything inside GOPATH on every run. There are legitimate use cases for
having a ton of non-Go files inside a folder in your GOPATH e.g. You're
writing a website in Go which uses a lot of external files like
images/templates/styles whatever, you'll probably want to store them in a
subfolder of your main package, which inevitably ends up in your GOPATH and
gets scanned every time goimports runs.
On Tuesday, April 28, 2015 at 3:46:46 PM UTC+1, Brad Fitzpatrick wrote:

Telling Linux to drop all its buffer & dentry caches before running it is
a more consistent way to reproduce.

Whenever I find goimports to be slow, inevitable I find that I have the
Linux kernel or Android or OpenOffice or something gigantic under my
$HOME/src.


On Tue, Apr 28, 2015 at 8:06 AM, David Crawshaw <craw...@golang.org
<javascript:>> wrote:
Before adding a layer I would like to understand how to reproduce this
slowness. With a large GOPATH on both linux and darwin I cannot get
goimports to take more than a fraction of a second. What is in your
GOPATH?

On Tue, Apr 28, 2015 at 7:26 AM, Alberto García Hierro
<alb...@garciahierro.com <javascript:>> wrote:
Hi,

While working on a project, I've noticed that goimports is very slow
whenever I used it on a file that references non-exported symbols found in
another file in the same package. After some digging, it looks like
goimports spends most of its time loading packages and listing
directories
(from imports.Process) and it does this on every invocation. This has also
been noticed by other projects like vim-go, which switched from using
goimports to gofmt by default, due to the issues caused by the long runtime
required by goimports (vim freezing for several seconds on save).

I propose to add an opt-in cache to tools/imports, so programs like
goimports might decide to use this cache.

Goals:
- Reduce time spent by programs relying on tools/imports, while still
providing correct results
- Cache must be opt-in
- Cache must be per-program (some programs might not need a cache since they
might do little processing or might not require fast execution)
- Setting up and saving the cache must the program's responsibility.
- Minimal overhead for callers not using the cache.

Non-goals:
- Forward compatibility. It's a cache, so future breaking changes to the
cache format might just discard the old one and rebuild it.

Discarded alternative solutions:
- Loading the current package and checking its non-exported symbols. There
are already many tools that feed goimports from stdin, so it has no way to
check the package containing the current input file. This solution would
also not alleviate the problem of non existing symbols (e.g. a mistyped
variable).

New exposed functions in tools/imports

- LoadCache(r io.Reader) error
- SaveCache(w io.Writer) error

With these two functions, each program is responsible for determining where
it wants the cache to be stored as well as loading and saving it. Using
interfaces rather than filenames allows also out-of-fs caching (e.g.
memcache, redis, whatever the caller wants it to be).

Potential gains with naive implementation:

I've implemented a proof of concept cache which uses the directory mtime to
check whether a directory needs to be scanned again or not. I do
understand
that this is not bullet proof, since changes to attributes in the files
won't update the directory mtime, keep in mind this is PoF.

All runs have been averaged between 5 times, after discarding the first run
(to minimize effects from in-kernel fs caching).

goimports (current implementation):
real 0m5.198s

goimports (with cache, cold run with cache not built yet):
real 0m5.972s

goimports (with cache populated):
real 0m0.110s

Please, let me know what you think about this proposal.

Regards,
Alberto

--
You received this message because you are subscribed to the Google Groups
"golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-dev+...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-dev+...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 16 | next ›
Discussion Overview
groupgolang-dev @
categoriesgo
postedApr 28, '15 at 11:26a
activeApr 28, '15 at 8:39p
posts16
users8
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase