Serialization in Convergence/Rappid/Jointjs

I’m trying to get Convergence working with rappid and jointjs. Thanks to @alec, I’ve got
initial functinality working great with diagram sharing.

My next important task is serialization. I’ve got a simple Load and Save function using localStorage working using new toolbar buttons and the following simple code.

   load: function () {
        //window.alert("Loading");
        let JsonString = localStorage.getItem('testjson');

          if (!JsonString) {
            window.alert("Nothing has been stored");
          }
          else {
              this.graph.fromJSON(JSON.parse(JsonString));
          }
      },

      save: function () {
        //window.alert("Saving");
        //var jsonObject = graph1.toJSON();
        var jsonObject = this.graph.toJSON();
        var jsonString = JSON.stringify(jsonObject);
        localStorage.setItem('testjson', jsonString);
      },

However, if I do a Convergence share, things break.*

I have skimmed the Convergence API docs and expect that the answer is buried in there somewhere, but I wonder if anyone here can give me a little guidance / pointers? @alec can you help?

I’m not sure exactly how the rappid/jointJS integration was built, but typically we listen to changes in a Convergence RealTimeModel and update the diagram accordingly.

You probably want the toJSON method on a realtimeobject, though.

Yes. I think that is correct. You can use the toJSON method to get the data to save. When you start up a new collaboration, you can create a new ephemeral model using that JSON Data.

HI Alec,
Can you give me a little more advice about how to use a realtimeobject? We’re trying to a low code / no code approach to this, so have limited developer resources, so I’m hoping someone out there can give me a bit more details. Do we know who authored the great rappid/convergence sample code?

My specific problem is this:

  1. I initialize a diagram using the demo code. Works great
  2. I press “share” and open in a new browser. Sharing works great.
  3. I run my save code, as above. No problem.
  4. I hit the “clear” button from the sample code. Everything clears on both my main browser as well as the remote (shared) one.
  5. I run the “load” code above. This is where there’s a problem: I can see the diagram in the main screen, but not in the remote view.

So the diagram isn’t re-initializing to the remote site. Looking for the mechanism by which the diagram is created in the first place seems to be the right approach. So I have:

    initializeCollaboration: function () {
      var collectionId = "diagrams";
      var activityId = collectionId + ":" + this.modelId;
      var chatId = collectionId + ":" + this.modelId;

      var self = this;

      var loadModel = self.domain.models()
        .openAutoCreate({
          id: this.modelId,
          collection: collectionId,
          ephemeral: true,
          data: DEFAULT_GRAPH_DATA
        })
        .then(function(model) {
            self._model = model;
            new GraphAdapter(self.graph, model);
        });

So I’m guessing I need to do some version of this for a load function, but I’m not sure where to go next. I haven’t, in particular, gone up the Backbone learning curve, so I’m not sure how to find self.domain.models in a new code segment, or if indeed there’s another approach I should be using.

TIA for any advice you can provide,
L

Thank you, @michael . I’m intrigued by the idea of using a realtimeobject. Not sure where the right object is in this demo app, though, nor how to recognize it if I’m to go on a hunt. Any pointers? -L

We should probably test to see if this works in our demo app or not. It might be something we overlooked. There are two possibilities. The first is that we are missing code all together to handle the load function. The second is that the load function may be interacting with the RealTimeModel in a way that we are not listening for on the remote end. My first thought would be to search through the code and find where the click on the “Clear” button is handled and see exactly what that does. Then I would look for the “Load” button and do the same. I will take a look today also and report my findings.

Thanks, Michael! Fyi, the load function isn’t in the demo app, I added load and save buttons plus the above code, myself. I look forward to what you come up with.

An I see. So when you are doing a load, I assume the RealTimeModel is still the same one (e.g. the same modelId.).

My guess is that the graph.fromJSON method is either not firing an event, or it is firing an event we are not listening for. Now that I see what you are doing I think I at least know where to look. I hadn’t put the the two together. The way the jointjs-utils works is to basically create a two way binding between a convergence model and a JointJS graph. To do this we listen to specific events on JointJS and then call methods on Convergence, and vice versa. It is possible that when you call fromJSON on the graph that it completely replaces the graph model that we are listening to. Such that we are no longer listening to the right data structures.

It might be that we need to update the json-utils project to handle the .fromJSON method on the graph. I think we should be able to test this directly in the jointjs-utils project without even needing to look at the rappid code.

I will take a look over the next day or so and let you know.

this sounds good, thanks @michael. Fyi the rappid/convergence demo code I’m modifying is pretty extensive, so recommend you test your solution in that.

I beleive most of the problem is related to this issue:

The following commit seems to resolve the problem in the jointjs-utils.

Thanks for this @michael, very much appreciated. I’m beginning the merge into the rappid/convergence demo code base I’ve been working with (not super easy because the js has been appended into a single giant library file…).

Three questions for you:

  1. Have you incorporated your change into the rappid/convergence demo code and tested a load function there? If so, I would be eternally grateful if you could share that with me; it could save me days of work.
  2. If not, can you recommend some driver code for me to use to test the sync? Re my load code above, what should change in order to get the sync to work. Did you add a load function to any of the examples that would demo this?
  3. Also assuming not, do you (or @alec ) have any recommendations for how I should go about incorporating your jointjs-utils merge? I see there are dependencies, so it’s not just a single library swap in. I was going to check your dependencies, start to split out libraries from the single-file form it’s in now, swap in new versions as appropriate, then check if there are any other library updates (super hard since, again, @alec sent it to me as a giant file, which was good for ensuring it was robust and worked, not so great for library updates). Hopefully the needed update is limited to jointjs-utils, but I’m concerned about other dependencies / mixed versions that haven’t been tested together.

Thank you again for your help, and in advance for any advice you can provide to me. fwiw, this will be used in an important new project, defining a discipline featured on cspan, npr, by Gartner, funded by the UK government, a sovereign bank of a G20 nation, a couple of silicon valley tech companies, and NASA. Apps include covid and climate change. It’s hard to imagine work more important than this :slight_smile:

Best,
L

Hi again @alec and @michael. As I’m starting to try to understand the demo code base vis a vis what you’ve got on github, I’m starting to suspect that the libraries it uses are much older versions than the latest. For instance, GraphAdapter in the demo code (which has no indicator of what version it’s from) is:


But on github it’s:

So can you please confirm I’m right / that the rappid/convergence graph demo (sent to me by @alec since we’re a rappid license owner) is not in sync with the latest libraries? If so, that’s going to imply that the fix that @michael sent is going to take a big effort to integrate.

Given that we’d allocated almost zero developer resources to this / we’re just trying to get a bare-bones prototype up and running (which nonetheless needs load and save functionality) this puts me in a pickle.

If my suspicions are right, I wonder if you guys could suggest a creative workaround? Is there some small change I could make to the existing demo code just to get load/save working? Then we can plan for a much bigger dev project down the road to update this code base to be in sync with the latest libraries (at which point there’s a good chance we’ll hire Convergence devs for the task). But I won’t be able to get there without a basic prototype.

Or am I missing something (I hope I am!)?
-L

@lypratt. For some history. The diagram editor demo was a proof of concept application that we developed quite a while ago over the course of a weekend. We had obtained a demo license of Rappid, which is a great library and created the demo application. This demo application is not a product of ours and isn’t published open source either since it contains the proprietary Rappid code, that requires a license.

When building out the demo, we just listened to the exact events from jointjs that the demo app itself caused to be emitted. If the demo app (as written) didn’t use some portion of the jointjs library, we didn’t add code to handle that. Similar to you, it was just a quick proof of concetp.

However, a large portion of the integration only needs jointjs. So we extracted the code we (quickly) wrote for the demo, refactored it and improved it for the jointjs-utils library on github. I don’t think we ever went back to the graph demo to re-factor it to actually use the jointjs-utils.

The demo (for our purposes) works as is and it was never a priority to go back and clean it up, since we had never shared it with anyone. I think the version of Rappid in that demo is also quite old. So it’s not that there is an “old version” of the jointjs-utils in this demo. It’s that this demo was the precursor to the jointjs utils.

Long term, if this is something you want to use, I would suggest that we updated it for both the Rappid version, and to use the jointjs-utils project. We should probably get the latest version of Rappid and their demo app from ClientIO and then re-integrate the chanced.

Looking at the code for the diagram editor demo, I don’t think it ever the fromJSON method (outside of loading the initial graph). I believe this triggers the “reset” event on the “cells” inside the graph. In the demo app, I don’t think we are listening to that, which is likely why nothing is happening when you load the graph. The GraphAdapter is not listening for that event.

To point you in the right direction… I looked at how to port my fix into this code and will suggest some edits below.

1 Like

First, find this code.

GraphAdapter.prototype = {
  init: function () {
    var data = this.model.root().value();
    this.graph.fromJSON(GraphAdapter.dataToGraphJson(data));
    this.cellsModel = this.model.elementAt("cells");

    this.graph.getCells().forEach(function (cell) {
      new CellAdapter(cell, this.cellsModel.get(cell.id));
    }.bind(this));

    // other code omitted
  }
}

Extract the last few shown lines into it’s own method:

this.graph.getCells().forEach(function (cell) {
   new CellAdapter(cell, this.cellsModel.get(cell.id));
}.bind(this));

So it will look like this:

GraphAdapter.prototype = {
  init: function () {
    var data = this.model.root().value();
    this.graph.fromJSON(GraphAdapter.dataToGraphJson(data));
    this.cellsModel = this.model.elementAt("cells");

    this.bindAllCells();

    // other code omitted
  },

  bindAllCells: function() {
    this.graph.getCells().forEach(function (cell) {
      new CellAdapter(cell, this.cellsModel.get(cell.id));
    }.bind(this));
  }
}

Then at the end of the GraphAdapter.init() function add this code.

    // Local graph reset.
    this.graph.on("reset", function (event) {
      if (!this.remote) {
        var cells = {};
        event.models.forEach(cell => {
          cells[cell.id] = cell.toJSON();
        });
        this.cellsModel.value(cells);

        this.bindAllCells();
      }
    }.bind(this));

    // remote graph reset.
    this.cellsModel.on("value", function (event) {
      this.remote = true;

      const cellsData = event.element.value();
      const cells = [];
      Object.keys(cellsData).forEach(id => {
        cells.push(cellsData[id]);
      });
      this.graph.resetCells(cells);

      this.bindAllCells();
      this.remote = false;
    }.bind(this));
1 Like

FYI. I tested this code in our demo app and it appears to work. Let us know if this is helpful.

1 Like

@michael : AWESOME, thank you for this background, and for your fast response, it’s really helpful, and helps me to understand how to move forward. I’ll work on incorporating your code today and let you know how it goes.

I’m impressed you wrote this in a weekend, it’s a great demo.