1
// Copyright 2018 The Go Authors. All rights reserved.
2
// Use of this source code is governed by a BSD-style
3
// license that can be found in the LICENSE file.
6
Package packages loads Go packages for inspection and analysis.
8
The [Load] function takes as input a list of patterns and returns a
9
list of [Package] values describing individual packages matched by those
11
A [Config] specifies configuration options, the most important of which is
12
the [LoadMode], which controls the amount of detail in the loaded packages.
14
Load passes most patterns directly to the underlying build tool.
15
The default build tool is the go command.
16
Its supported patterns are described at
17
https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns.
18
Other build systems may be supported by providing a "driver";
19
see [The driver protocol].
21
All patterns with the prefix "query=", where query is a
22
non-empty string of letters from [a-z], are reserved and may be
23
interpreted as query operators.
25
Two query operators are currently supported: "file" and "pattern".
27
The query "file=path/to/file.go" matches the package or packages enclosing
28
the Go source file path/to/file.go. For example "file=~/go/src/fmt/print.go"
29
might return the packages "fmt" and "fmt [fmt.test]".
31
The query "pattern=string" causes "string" to be passed directly to
32
the underlying build tool. In most cases this is unnecessary,
33
but an application can use Load("pattern=" + x) as an escaping mechanism34
to ensure that x is not interpreted as a query operator if it contains '='.
36
All other query operators are reserved for future use and currently
37
cause Load to report an error.
39
The Package struct provides basic information about the package, including
41
- ID, a unique identifier for the package in the returned set;
42
- GoFiles, the names of the package's Go source files;
43
- Imports, a map from source import strings to the Packages they name;
44
- Types, the type information for the package's exported symbols;
45
- Syntax, the parsed syntax trees for the package's source code; and
46
- TypesInfo, the result of a complete type-check of the package syntax trees.
48
(See the documentation for type Package for the complete list of fields
49
and more detailed descriptions.)
53
Load(nil, "bytes", "unicode...")
55
returns four Package structs describing the standard library packages
56
bytes, unicode, unicode/utf16, and unicode/utf8. Note that one pattern
57
can match multiple packages and that a package might be matched by
58
multiple patterns: in general it is not possible to determine which
59
packages correspond to which patterns.
61
Note that the list returned by Load contains only the packages matched
62
by the patterns. Their dependencies can be found by walking the import
63
graph using the Imports fields.
65
The Load function can be configured by passing a pointer to a Config as
66
the first argument. A nil Config is equivalent to the zero Config, which
67
causes Load to run in LoadFiles mode, collecting minimal information.
68
See the documentation for type Config for details.
70
As noted earlier, the Config.Mode controls the amount of detail
71
reported about the loaded packages. See the documentation for type LoadMode
74
Most tools should pass their command-line arguments (after any flags)
75
uninterpreted to [Load], so that it can interpret them
76
according to the conventions of the underlying build system.
78
See the Example function for typical usage.
82
[Load] may be used to load Go packages even in Go projects that use
83
alternative build systems, by installing an appropriate "driver"
84
program for the build system and specifying its location in the
85
GOPACKAGESDRIVER environment variable.
87
https://github.com/bazelbuild/rules_go/wiki/Editor-and-tool-integration
88
explains how to use the driver for Bazel.
90
The driver program is responsible for interpreting patterns in its
91
preferred notation and reporting information about the packages that
92
those patterns identify. Drivers must also support the special "file="
93
and "pattern=" patterns described above.
95
The patterns are provided as positional command-line arguments. A
96
JSON-encoded [DriverRequest] message providing additional information
97
is written to the driver's standard input. The driver must write a
98
JSON-encoded [DriverResponse] message to its standard output. (This
99
message differs from the JSON schema produced by 'go list'.)
101
package packages // import "golang.org/x/tools/go/packages"
105
Motivation and design considerations
107
The new package's design solves problems addressed by two existing
108
packages: go/build, which locates and describes packages, and
109
golang.org/x/tools/go/loader, which loads, parses and type-checks them.
110
The go/build.Package structure encodes too much of the 'go build' way
111
of organizing projects, leaving us in need of a data type that describes a
112
package of Go source code independent of the underlying build system.
113
We wanted something that works equally well with go build and vgo, and
114
also other build systems such as Bazel and Blaze, making it possible to
115
construct analysis tools that work in all these environments.
116
Tools such as errcheck and staticcheck were essentially unavailable to
117
the Go community at Google, and some of Google's internal tools for Go
118
are unavailable externally.
119
This new package provides a uniform way to obtain package metadata by
120
querying each of these build systems, optionally supporting their
121
preferred command-line notations for packages, so that tools integrate
122
neatly with users' build environments. The Metadata query function
123
executes an external query tool appropriate to the current workspace.
125
Loading packages always returns the complete import graph "all the way down",
126
even if all you want is information about a single package, because the query
127
mechanisms of all the build systems we currently support ({go,vgo} list, and128
blaze/bazel aspect-based query) cannot provide detailed information
129
about one package without visiting all its dependencies too, so there is
130
no additional asymptotic cost to providing transitive information.
131
(This property might not be true of a hypothetical 5th build system.)
133
In calls to TypeCheck, all initial packages, and any package that
134
transitively depends on one of them, must be loaded from source.
135
Consider A->B->C->D->E: if A,C are initial, A,B,C must be loaded from
136
source; D may be loaded from export data, and E may not be loaded at all
137
(though it's possible that D's export data mentions it, so a
138
types.Package may be created for it and exposed.)
140
The old loader had a feature to suppress type-checking of function
141
bodies on a per-package basis, primarily intended to reduce the work of
142
obtaining type information for imported packages. Now that imports are
143
satisfied by export data, the optimization no longer seems necessary.
145
Despite some early attempts, the old loader did not exploit export data,
146
instead always using the equivalent of WholeProgram mode. This was due
147
to the complexity of mixing source and export data packages (now
148
resolved by the upward traversal mentioned above), and because export data
149
files were nearly always missing or stale. Now that 'go build' supports
150
caching, all the underlying build systems can guarantee to produce
151
export data in a reasonable (amortized) time.
153
Test "main" packages synthesized by the build system are now reported as
154
first-class packages, avoiding the need for clients (such as go/ssa) to
155
reinvent this generation logic.
157
One way in which go/packages is simpler than the old loader is in its
158
treatment of in-package tests. In-package tests are packages that
159
consist of all the files of the library under test, plus the test files.
160
The old loader constructed in-package tests by a two-phase process of
161
mutation called "augmentation": first it would construct and type check
162
all the ordinary library packages and type-check the packages that
163
depend on them; then it would add more (test) files to the package and
164
type-check again. This two-phase approach had four major problems:
165
1) in processing the tests, the loader modified the library package,
166
leaving no way for a client application to see both the test
167
package and the library package; one would mutate into the other.
168
2) because test files can declare additional methods on types defined in
169
the library portion of the package, the dispatch of method calls in
170
the library portion was affected by the presence of the test files.
171
This should have been a clue that the packages were logically
173
3) this model of "augmentation" assumed at most one in-package test
174
per library package, which is true of projects using 'go build',
175
but not other build systems.
176
4) because of the two-phase nature of test processing, all packages that
177
import the library package had to be processed before augmentation,
178
forcing a "one-shot" API and preventing the client from calling Load
179
in several times in sequence as is now possible in WholeProgram mode.
180
(TypeCheck mode has a similar one-shot restriction for a different reason.)
182
Early drafts of this package supported "multi-shot" operation.
183
Although it allowed clients to make a sequence of calls (or concurrent
184
calls) to Load, building up the graph of Packages incrementally,
185
it was of marginal value: it complicated the API
186
(since it allowed some options to vary across calls but not others),
187
it complicated the implementation,
188
it cannot be made to work in Types mode, as explained above,
189
and it was less efficient than making one combined call (when this is possible).
190
Among the clients we have inspected, none made multiple calls to load
191
but could not be easily and satisfactorily modified to make only a single call.
192
However, applications changes may be required.
193
For example, the ssadump command loads the user-specified packages
194
and in addition the runtime package. It is tempting to simply append
195
"runtime" to the user-provided list, but that does not work if the user
196
specified an ad-hoc package such as [a.go b.go].
197
Instead, ssadump no longer requests the runtime package,
198
but seeks it among the dependencies of the user-specified packages,
199
and emits an error if it is not found.
201
Overlays: The Overlay field in the Config allows providing alternate contents
202
for Go source files, by providing a mapping from file path to contents.
203
go/packages will pull in new imports added in overlay files when go/packages
204
is run in LoadImports mode or greater.
205
Overlay support for the go list driver isn't complete yet: if the file doesn't
206
exist on disk, it will only be recognized in an overlay if it is a non-test file
207
and the package would be reported even without the overlay.
212
They are not portable concepts, but could be made portable.
213
Our goal has been to allow users to express themselves using the conventions
214
of the underlying build system: if the build system honors GOARCH
215
during a build and during a metadata query, then so should
216
applications built atop that query mechanism.
217
Conversely, if the target architecture of the build is determined by
218
command-line flags, the application can pass the relevant
219
flags through to the build system using a command such as:
220
myapp -query_flag="--cpu=amd64" -query_flag="--os=darwin"
221
However, this approach is low-level, unwieldy, and non-portable.
222
GOOS and GOARCH seem important enough to warrant a dedicated option.
224
- How should we handle partial failures such as a mixture of good and
225
malformed patterns, existing and non-existent packages, successful and
226
failed builds, import failures, import cycles, and so on, in a call to
229
- Support bazel, blaze, and go1.10 list, not just go1.11 list.
231
- Handle (and test) various partial success cases, e.g.
232
a mixture of good packages and:
236
packages with malformed package or import declarations
241
Make sure we record errors at the correct place in the graph.
243
- Missing packages among initial arguments are not reported.
244
Return bogus packages for them, like golist does.
246
- "undeclared name" errors (for example) are reported out of source file
247
order. I suspect this is due to the breadth-first resolution now used
248
by go/types. Is that a bug? Discuss with gri.