archives

NDArray/multi-columnar with efficient CRUD operations?

I'm in the hunt for material or implementations of NDArray and/or columnar structures with not-bad support for CRUD operations (insert-update-delete) that could work in-memory.

I'm aware of kdb+ but my understanding is only for big append-only loads then calculations on it.

Currently, I have for my own little relational lang (http://tablam.org) several structures that backed the relations (BTreeMaps - Vectors - Table - Scalars). The main is a table:

https://bitbucket.org/tablam/tablam/src/41b3b0676d6062031998af48b683f8f0062bf279/core/src/types.rs#lines-221

#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Ord, Hash)]
pub struct Table {
pub schema: Schema,
pub data: Vec,
}

and this is ok. But wonder what other options I could explore. (The allure for me in use just a NDArray-like container is that I can reduce my implementations to just 2 BtreeMaps - NDArray)

Rope

This is my very first post on this site. Also this is my first post regarding my new PL idea: Rope

First, an introduction to me (in a hopefully non-narcissistic way):
Skip over this if you want to avoid hearing someone talk about themselves
My name is Jocob (it was a typo from Jacob; we didn't learn about it until I was 19 and by then I was a legal adult so I had the choice to keep it). I've been programming for about 5 years. First Excel macros, then C# in Unity, and from there I decided to get a career in the field. Now I'm a "DevOps Engineer" (I don't know why they prepended a paradigm onto a job title). Professionally, I work mainly with Python, HTML/JavaScript/CSS, PowerShell, Bash, and increasingly GoLang and Ruby. Personally, I like to use C# both for Windows Apps and for Unity. I'm also building a site on AWS using Django/Bootstrap/PostgreSQL.

I have no experience or education designing programming languages.

If I have seen further than others, it is by standing upon the shoulders of giants. - Isaac Newton

I get it, and I agree with it for the most part; why waste your time doing what has already been done? What progress would we make if everyone started from how to make fire and tried making it to the moon? But there is something that I don't agree with. (Not with Newton, but with a secondary application) Should we all only blaze trails where a previous trailblazer left off? My personal experience has taught me that, if you disagree, then you'll learn more and, rarely, discover something amazing. It is with that mentality that I want to introduce "Rope."

Second, an introduction to ROPE:
Skip over this if you get annoyed reading about people's naive hopes and dreams
The idea is simple: I want a "language to create languages." Here are the main features I'm aiming for:

  • Doesn't inherit from Fortran: I want the language itself to help anyone understand exactly how they're instructing a machine to operate, so I want it to inherit directly from machine language. The dream is that someone without a computer science degree could understand how ones and zeros translate to a UI on a monitor if they took the time to read the source code.
  • Intention over implementation: Each basic function will have an input/output criteria with a standard implementation. But if someone can propose a better, faster implementation, then that function can be replaced without impacting existing code. This would mean, ideally, that people would never need to worry about losing functionality or rewriting their code in the event of an update. They could just update, and things would work better.
  • Both a Script and a Markup Language: While someone would be able to write any application they want in Rope, they would primarily use it to define a higher level language. I'd hope to eventually create ports for other major languages that would allow Rope to import and export to them. Subsequently, this would enable a "translate" feature that recreates scripts in another language. (Substituting raw Rope syntax wherever there is no translation, which would indicate to the Rope developer that the port needs to be improved.
  • Prepackaged IDE, Compiler, and Interpreter: Rope would come with everything you need to write and play right out of the box. I know personally that the faster I can see the output of my code, the quicker I can learn it. The compiler would be able to produce an executable file if you write as a script, or it could produce another compiler if you write as markup. The interpreter behaves the same as the compiler, but could also read and run ported languages as long as their is a markup file exists. This would, theoretically, allow for non-interpreted languages to be used live scripts.
  • Visually incarnatable: Because the core of the language would be simple, and to help make the language easier to use in other Countries, Rope would be able to be represented entirely visually.
  • Built-in basic kernel: To help with being cross-platform, and because Rope aims to be able to directly bridge human and machine, Rope would have an ability to interact directly with hardware. Markups would be written to allow it to read device drivers for various platforms. Ideally this would mean that it could use *Nix drivers and Windows drivers at the same time.
  • Community linked at every aspect: Each implementation, for example the "Output to Console" implementation, would have both an "intention ID" and an "implementation ID." The IDE would integrate with an online resource that allows people to discuss and collaborate on each piece of the language itself. You could, in this example, click on the "print" (or whatever it will be called) function, and see the discussion and versioning of that core function. It would allow the language to grow and be discussed much more rapidly than through mediums like GitHub, because it could be done in the middle of coding. (Though that may be distracting...) There would also be a like/dislike voting system for everything, as well as "Implementation Alternates" thought could be seamlessly swapped out during development and then automatically aggregate bench-marking from all willing participants.
  • Syntax to define syntax: This is a necessary part of the "language to create languages" goal, but there's no reason why that couldn't be used within the code itself to make things easier. For example, someone could want one script with lots of commenting, and another separate script with lots of string literals. Rope would allow them to configure each script at the start to handle those syntactical preferences.

That's all I got for now. But I'm sure I'll come up with more. The lofty goal behind the "language to create languages" philosophy is that, at the start, everyone tries to make their PL perfectly suited for its purpose. But as we've seen, languages keep expanding and descendants get spawned as a result of either success or failure. So if we had to do it all over again, wouldn't we want a language that intended to have descendants. Wouldn't it have been nice to have a language that lived for its children? That's the idea I want to explore.

Oh and as a final thought. The origin of the name is this:
Recall
Observe
Prove
Express
In terms of how information is handled, it essentially means that it can pull from either memory or the outside world, process it, and then output it back into the world. The name is meant to represent every state of information handling, because that's essentially what a computer is, and be a snappy acronym as well. In regards why "create" isn't included in my theory of information handling, there's actually a philosophical reason for that. I don't think anything in human memory comes from absolute nothingness, and that creativity is probably an illusion.