elasticsearch representation has bespoke keys
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| charmworld |
Fix Released
|
High
|
Abel Deuring | ||
Bug Description
Elasticsearch is not a free-form data store; it uses a mapping, and that mapping has an entry for every unique key it encounters. This prevents pre-defining value types, which in turn leads to issues like bug #1176452. It also means that the mapping is so long that it is not human-readable. (The mapping for 10 charms turns out to be 1598 lines, pretty-printed.)
Instead, the representation given to elasticsearch should make all user-defined values *values*, not keys. For example:
{'foo': 'bar', 'baz': 'qux'} => [{'name': 'foo', 'value': 'bar'}, {'name': 'foo', 'value': 'qux'}]
Currently affected fields: "config.options" (but work in progress), "files", "provides", "requires", "peers".
When fixing files, it may make sense to address bug #1189992 at the same time.
Related branches
- Abel Deuring (community): Approve
- Benji York (community): Approve
-
Diff: 229 lines (+89/-42)4 files modifiedcharmworld/migrations/versions/019_fixed_mapping_for_interfaces.py (+10/-0)
charmworld/migrations/versions/tests/test_migrations.py (+44/-0)
charmworld/search.py (+30/-20)
charmworld/tests/test_search.py (+5/-22)
| description: | updated |
| Changed in charmworld: | |
| status: | New → Triaged |
| importance: | Undecided → High |
| tags: | added: elasticsearch |
| Curtis Hovey (sinzui) wrote | #1 |
| Abel Deuring (adeuring) wrote Re: [Bug 1194907] Re: elasticsearch representation has bespoke keys | #2 |
| Curtis Hovey (sinzui) wrote | #3 |
| Changed in charmworld: | |
| assignee: | nobody → Abel Deuring (adeuring) |
| status: | Triaged → Fix Committed |
| Changed in charmworld: | |
| status: | Fix Committed → Fix Released |
Work on this bug has stalled. I don't think this is fixed because We need a list of the remaining user-created keys? I think options is fixed. The comments in charmworld.search imply that requires and provides use dynamic mapping. The files bug is still open.