Chapter 8. Other Techniques 1. Frequently required features in Rails

This chapter introduces some implementation policies for frequently required features in Rails.

Updating a single Model

Use form_with in view. Also, implement Controller and Model according to it. Scaffold can generate a basic code, and you can add and modify it as needed.

Rails Guides - Dealing with Model Objects

Updating with child Model(s) (1:1, 1:N)

Implementation ways are the following.

Use accepts_nested_attributes_for
Use autosave: true
Save for each Model (without above ways)

`accepts_nested_attributes_for`

accepts_nested_attributes_for, autosave: true perform child Models validation, save, and transaction when the parent Model saved. It can be used in both 1:1 and 1:N cases. accepts_nested_attributes_for also can use nested parameter.

If these are not used, the child Model(s) are not updated when the parent Model are updated. In such a case, need to save each Models within transaction block.

In the case of 1:1 or even 1:n with fixed record size are is relatively simple, but 1:n with flexible record size (has add/remove child) will be difficult. Because the parameter patterns increases and you need to use add/delete forms with Javascript, which increases the implementation difficulty. Although, the difficulty is much the same even without accepts_nested_attributes_for.

Rails Guides - Building Complex Forms

Parameter Structure

For example, if

Parent table is users and
Child table is user_contracts

then, the following will be suitable.

1:1 then params[:user][:user_contract_attributes][:name]
1:N then params[:user][:user_contracts_attributes][{index}][:name]

Parameters written in Hash will look like the following.

# 1:1
{
  user:
    name: 'Jhon',
    {
      user_contract_attributes:
      {
        id: 1,
        started_on: '1/1/2023',
        finished_on: '1/31/2023'
      }
    }
}

# 1:N
{
  user:
    name: 'Jhon',
    {
      user_contracts_attributes:
      [
        {
          id: 1,
          started_on: '1/1/2023',
          finished_on: '1/31/2023'
        },
        {
          id: 2,
          started_on: '2/1/2023',
          finished_on: '2/28/2023'
        },
        {
          id: 3,
          _destroy: '1'
        }
      ]
    }
}

The user_contracts_attributes contents changes as follows.

For new record, id is not required.
For update, id is required.
For destroy, _destroy is required. Also, you have to set allow_destroy: true option to accepts_nested_attributes_for .

Use fields_for in View to control these parameter hierarchies.

Rails API - fields_for

If the parameter hierarchies does not fit with the frontend, can use autosave: true.

How to proceed (how to isolate the problem)

When using accepts_nested_attributes_for, you may proceed as follows.

Implement the server side alone (no View) can update data if the correct parameters are sent. And make a request spec on that.
Implement the View, and check the HTML source to make sure the parameters are correct.
Check the Rails development log to confirm that the correct parameters are submitted and permitted correctly.

This will make it easier to determine what is the problem.

accepts_nested_attributes_for is a useful, but it takes a little time to understand. If you want to prioritize the development speed due to the release date is near, you may not choose to use accepts_nested_attributes_for.

Rough code for multi-model update

Using `accepts_nested_attributes_for`

# Set nested parameter
model1.assign_assributes(nested_params)

# Validate (all child Models are validated at the same time)
if model1.invalid?
  # Handle validation errors
end

# Update (all child Models are updated at the same time)
model1.save!

Note that accepts_nested_attributes_for include internally autosave: true.

Using `autosave: true`

# Set each parameter
model1.assign_assributes(model1_params)
model1.child_model.assign_assributes(child_model_params)

# Validate (attributes set child Models are validated at the same time)
if model1.invalid?
  # Handle validation errors
end

# Update (all child Models are updated at the same time)
model1.save!

Without `accepts_nested_attributes_for` and `autosave: true`.

# Set each parameter
model1.assign_assributes(model1_params)
model2.assign_assributes(model2_params)

# Validate each Models
if model1.invalid?
  # Handle validation errors
end

if model2.invalid?
  # Handle validation errors
end

# Update each Models within transaction (will be able to rollback on one side error)
Model1.transaction do
  model1.save!
  model2.save!
end

There are not many gems that could be used.

I don't know of any gem as an alternative that could be used.

The gem cocoon if using jQuery. For the add/remove form.
The gem nested_form is deprecated, so I don't recommend it.

Many-to-Many (N:N) Associations

There is a way to use has_and_belongs_to_many.

Rails Guides - The has_and_belongs_to_many Association

However, I don't remember having implemented N:N associations for a while now, so I can't judge whether this method is easy to use or not. I don't think N:N associations are very easy to implement in Rails, it might be good to first consider whether it is possible to implement it without N:N associations.

Intermediate table name

Use which one of the following.

Connected table names
Implied the entity

e.g. If you have tables named users and magazines.

In case 1, it can be users_magazines. It is easy to understand the relation, but it is not intuitive what the entity is.

In case 2, it can be subscriptions. It is easy to understand the entity, but a little harder to know what the table related.

Which one is better

In my opinion, it should be thought based on the importance of the table as follows.

If the importance of subscriptions are high in your system, you should choose it.
If it is just an accessory information of user, you can choose users_magazines.

If you can't decide it, ask your team members for their opinions.

There are gems devise and sorcery for login , and either of these is often used. It can be implemented login feature relatively easy.

Since devise has a longer history and more information, it may be a better choice for those who are not that familiar with session management.

Some people choose sorcery because they think that devise is difficult to customize, so if you have special requires with login, you should check the implementation way before deciding Gem.

By the way, I don't remember having any trouble using devise.

Github - devise

Github - sorcery

The gem omniauth is often used, and additional gems for each service is needed (such as omniauth-google-oauth2, omniauth-twitter).

The basic implementation is not difficult relatively and there is a lot of information available, but it may be a little difficult for those who have never used OAuth to understand how it works. OAuth is not exclusive to Rails, so it may be useful to deepen your knowledge of it.

Note that it will be needed to register for each developer service in order to use SNS authentication.

Test in spec

Use OmniAuth.config.test_mode = true to ensure that no external HTTP requests are generated.

Github - omniauth

CSV File Read / Write

Use CSV, a standard class.

Read (Import)

Use CSV.foreach or CSV.read to implement it relatively easy.

Rubydoc - CSV#read

Rubydoc - CSV#foreach

Since you can get a lot of information on how to use it by Googling, I will omit the explanation here.

There seems to be a gem smarter_csv for easier, but I have never used it because I think the CSV class is enough. You can check it if interested.

And the gem roo is used for manipulating styles or multiple sheets.

Write (Export)

Use CSV.generate and send_data. There is also a lot of information available on how to implement Controller, so I think you can do as you like, but my best way is as follows.

Qiita - CSVダウンロード機能を実装

* Japanese article

A brief summary is as follows.

Request in CSV from View with like users_path(format: :csv).
Set @csv_data_variables and use send_data render_to_string in Controller, and use respond_to to devide by the format.
Put index.csv.ruby ({action_name}.csv.ruby) instead of View file, and use CSV.generate in it.

When 1, If you want to use the form tag, write the format as follows.

<%= form_with url: csv_export_path(format: :csv), local: true, method: :get do |f| %>
  (abbreviation)
<% end %>

File upload

ActiveStorage

It is the standard function and relatively easy to use, and there is a lot of information on it, so I omit the explanation here.

MinIO(for Amazon S3)

MinIO is highly recommended when using Amazon S3 as the upload destination, as it works almost the same as S3, making it easy to test.

dev.to - Using minio to mock S3 in rails test and development

File upload form requires `multipart: true`

Since form.file_field automatically adds multipart: true to the form attribute, it is easy to forget to add it when you implement as a different way such as rendering <input type='file' /> by javascript.

If it is not be able to uploaded files, the file parameter class may be String, not ActionDispatch::Http::UploadedFile, so you can notice multipart: true is missing.

Cryptographic hashing and encryption/decryption

Terms you should know

Cryptographic hashing Irreversible conversion from the original string to a seemingly random string called a hash value, different from Ruby Hash{key: :value}.
Encryption Reversible conversion from the original string to a seemingly random string.
Decryption The conversion from the encrypted string back to the original string using a decryption key.
Salt A string that is concatenated with the original string during hashing/encryption to make it longer and harder to crack the original string. Each user has a different value.
Hashing algorithm A type of hashing. For example, MD5 has a fairly low security level; SHA-256 is strong enough to be used in blockchains. There are many others.

Implementation

Hashing

The gem bcrypt is often used. The gem devise also depends on bcrypt. Its implementation is easier than the encryption, and may be implemented with standard functions (such as Digest::SHA256.hexdigest, SecureRandom).

Github - bcrypt-ruby

Encryption

It seems that it is not impossible to implement it using the standard functions of Rails and Ruby, but some web articles say that it is easier to use a gem.

I have implemented it using the gem attr_encrypted, and it was relatively easy, but it is currently not well maintained. I am not familiar with the current gems, but the gem lockbox is not bad.

Github - lockbox

What to Consider in Encryption Requirements

Personal and Critical Information

The definition of important information is different from each company. There may be companies that do not encrypt emails or ip addresses that are not linked to user name and address. It is necessary to first confirm what is the requirement.

It should be good to say "the risk is controlled because it is encrypted", even if there is unauthorized access to the DB server.

Recently, it seems that some companies are considering it as a risk to have personal information in their own company's DB, and are becoming increasingly conscious of the need to avoid it as much as possible. If possible, it should be managed personal information using an external secure service.

You may want to learn a little bit about encryption.

I think it is better to have knowledge somewhat about such sensitive matters, instead of just assuming it is OK because it works.

Also, if the person defining the requirements is not familiar with such things, the engineers should advise to them.

Payment

It is one of the most difficult things todo in Rails development In order to proceed as planned, it might be better to divide the individual phases like a waterfall (such as requirement definition, technical research, design and implementation, black box testing, security diagnosis, etc). In particular, implementation is highly dependent on requirements such as gems and APIs.

I don't know about overseas laws, but it may be basically used payment services such as Stripe or Paypal, and use the gem or API. For larger companies, it may be necessary to confirm with the legal department in advance.

Selection of services to be used

There is no such thing as "Just choose this one.", and the selection of a payment service is likely to be based on the following criteria.

Supported payment methods
- Credit card payment, e-money, convenience store payment, bank transfer, etc.
- Whether scalable to more payment methods in the future.
Whether have good tools such as gem, API, platform and it's documentation and support.
Whether the service is reliable
- Whether have in-depth knowledge of gem(API), platform and security, etc.
- Response speed for inquiries and problems, and business hours
Commission fee

Sending email

Use ActionMailer, a standard class. ActionMailer can be handled a mail as object, such as the mail body can be checked with .body.

Rails Guides - Action Mailer Basics

Sending mail is a relatively heavy process, so it might be better to make it as a background job to avoid timeout, reduce load, and manage retries.

Debugging an email

The gem letter_opener_web, letter_opener is very useful and I highly recommend it. They will be able to confirm the message and recipient, etc without actually sending it. The letter_opener_web will be able to check email in your browser, but it has not been updated for a while, so if you are concerned about it, use only the letter_opener.

Github - letter_opener

Github - letter_opener_web

Search (for admin)

In not use frontend framework, I highly recommend the gem ransack in admin.

Github - ransack

It may be a little difficult to write at first, but you will be able to write searches and sorts with much less code and at a blazing speed. It is an extension of ActiveRecord, so if you have trouble, you can use the original Rails functionality (Model scope) to implement it, and since it has been around for a long time, there is a lot of information.

HTTP communications

Use Net::HTTP, a standard class.

Ruby-Doc.org - Net::HTTP

The gem faraday seems to be used for easier implementation.

Github - faraday

update all

insert_all / update_all / upsert_all

These methods can be used for bulk insert or update.

Since validations and callbacks are not executed, they are not suitable for updating by user input. If validations are required, all data is checked beforehand with model.valid? and callbacks will be executed manually later if necessary. Sometimes, it may be easier to use background jobs.

activerecord-import

Since the these methods were added in Rails 6, sometimes the gemactiverecord-import was used before Rails6, but if you have no reason, you should use the standard functions such as insert_all and upsert_all instead.

Manipulating large data

Background job (asynchronous processing)

Use ActiveJob or gems such as Sidekiq, Resque and Delayed Job, but Sidekiq seems to be the most major. In addition, use Redis for job queue management.

Github - sidekiq

Place jobs in app/jobs. There are two implementation ways.

Use ActiveJob with inheriting ActiveJob::Base is used for the simple processing.
Use Sidekiq with include Sidekiq::Worker for using Sidekiq functions (such as sidekiq_options or sidekiq_retries_exhausted).

In case 1, use perform_async for async and .new.perform for sync.
In case 2, use perform_later for async and perform_now for sync.

Asynchronous processing is difficult to debug, so test it with sync method, or implement main processing as another class from the job and test it. If there are implementation policies for the service class, it is better to follow them to read easier.

find_each

Use find_each saves more memory than each for processing thousands of records.

each or to_a to scope allocates memory for all instances, but find_each is processed each 1000 instances internally. If it is not heavy processing, this method is enough.

find_in_batches

While find_each processes 1000 records internally, find_in_batches can get 1000 records as a Array, so it is possible to insert a sleep, logging or other processing for each batch.

Each just 1 second sleep makes a big difference in load.

Multithreading (concurrent processing)

Use Thread for high processing speed.

However, you should refrain from using it unless you really need it. Because of the difficulty of debugging and dealing with problems that may occur. If you use, it is recommended to use it only for the minimum range that takes the most time.

Scheduled job

Since Sidekiq alone cannot do something like "run every Monday at 1:00", use the gem Sidekiq-cron to do it. There are its conflict sidekiq-scheduler and clockwork.

Github - sidekiq-cron

Retry

Implement as Sidekiq jobs, and use sidekiq_options and sidekiq_retries_exhausted.

dev.to - Sidekiq's sidekiq_retries_exhausted hook